Expandable large language model jailbreak attack method and device, medium and product

Abstract

The invention relates to an expandable large language model jailbreak attack method and device, a medium and a product, and the method comprises the steps: obtaining a first prompt corresponding to a jailbreak task, and generating first answer data of the first prompt according to a question template; according to role description and / or situation description corresponding to the jailbreak task and a preset format requirement, writing content in a preset first prompt template is updated; by taking the first answer data as an example, transferring the first prompt in combination with the role description and / or the situation description in a first prompt template to obtain a second prompt meeting the format requirement; and obtaining second answer data generated by the target large language model based on the second prompt. By adopting the method, the problem that the assessment range of the security boundary is narrow when a large language model deals with the jailbreak attack can be solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a scalable large language model jailbreak attack method, device, medium and product. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, large language models (LLMs) have been widely used in finance, medical care, education and other fields due to their excellent natural language processing capabilities. However, although model developers have added many security mechanisms before the model is deployed, in certain situations, the model still has the risk of outputting harmful and toxic content, and jailbreaking of large language models is one of them. In the case of jailbreaking of large language models, the model's security protection mechanism can be bypassed through carefully designed inputs, inducing the model to generate harmful and malicious content, thereby causing model content security risks.

[0003] With respect to the problem in related technologies that the evaluation scope for determining the security boundary of a large language model in response to jailbreak attacks is narrow, no effective solution has been proposed so far. Summary of the invention

[0004] Based on this, it is necessary to provide a scalable large language model jailbreak attack method, device, medium and product that can solve the problem of narrow assessment range of security boundaries of large language models when responding to jailbreak attacks, in response to the above-mentioned technical problems.

[0005] In a first aspect, a scalable large language model jailbreak attack method is provided in this embodiment, and the method includes:

[0006] Obtaining a first prompt corresponding to the jailbreak task, and generating first answer data for the first prompt according to a question template;

[0007] According to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements, updating the written content in the preset first prompt template;

[0008] Taking the first answer data as an example, the first prompt is transcribed in combination with the role description and / or situation description in the first prompt template to obtain a second prompt that meets the format requirements;

[0009] Obtain second answer data generated by the target large language model based on the second prompt.

[0010] In some embodiments, after obtaining the second answer data generated by the target large language model based on the second prompt, the method further includes:

[0011] Obtaining a feedback parameter according to a matching condition between the second answer data and the jailbreak task;

[0012] Based on the feedback parameter, the written content in the first prompt template is updated to obtain a feedback prompt;

[0013] Taking the first answer data as an example, the second prompt is transcribed according to the role description and / or situation description, the feedback prompt and the first prompt to obtain a third prompt that meets the format requirement;

[0014] Obtain third answer data generated by the target large language model based on the third prompt.

[0015] In some embodiments, obtaining feedback parameters according to the matching between the second answer data and the jailbreak task includes:

[0016] Obtaining at least two second prompt templates with different judgment criteria according to the jailbreak task;

[0017] Obtaining a consistency parameter between the second prompt and the first prompt according to the second prompt template associated with the discrimination criterion and the prompt transcription;

[0018] obtaining evaluation parameters of the second answer data according to a second prompt template associated with the discrimination criterion and the jailbreak effect;

[0019] The feedback parameter is obtained according to the consistency parameter and the evaluation parameter.

[0020] In some of the embodiments, the judgment criteria in the second prompt template include one or more of the following: the degree of consistency between the semantics of the second prompt and the semantics of the first prompt, the degree of completion of the jailbreak task by the second answer data, the degree of concealment of the second prompt relative to the first prompt, and the degree of matching of the second answer data with format requirements.

[0021] In some embodiments, obtaining feedback parameters according to the matching between the second answer data and the jailbreak task includes:

[0022] Determining whether the second answer data matches the jailbreak task;

[0023] In the case that the second answer data does not match the jailbreak task, a feedback parameter is obtained according to a matching condition between the second answer data and the jailbreak task.

[0024] In some embodiments, before determining whether the second answer data matches the jailbreak task, the method further includes:

[0025] Determining whether the update number of the first prompt template reaches a preset number of iterations;

[0026] When the update times of the first prompt template does not reach the iteration times, a step of determining whether the second answer data matches the jailbreak task is performed.

[0027] In some embodiments, updating the written content in the preset first prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements includes:

[0028] Obtaining transcription prompt data according to one or more of the example data, transcription rule data, and prompt data corresponding to the jailbreak task;

[0029] The written content in the first prompt template is updated according to the role description and / or situation description, the preset format requirement and the transcription prompt data.

[0030] In a second aspect, in this embodiment, a scalable large language model jailbreak attack device is provided, the device comprising:

[0031] a jailbreak target generation module, configured to obtain a first prompt corresponding to the jailbreak task, and generate first answer data for the first prompt according to a question template;

[0032] An optimization instruction construction module, used for updating the written content in the preset first prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements;

[0033] A transcription output generation module, used for taking the first answer data as an example, transcribing the first prompt in combination with the role description and / or situation description in the first prompt template, to obtain a second prompt that meets the format requirements;

[0034] The jailbreak output generation module is used to obtain second answer data generated by the target large language model based on the second prompt.

[0035] In a third aspect, a computer device is provided in this embodiment, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the scalable large language model jailbreak attack method described in the first aspect is implemented.

[0036] In a fourth aspect, a computer-readable storage medium is provided in this embodiment, on which a computer program is stored. When the computer program is executed by a processor, the scalable large language model jailbreak attack method described in the first aspect is implemented.

[0037] In a fifth aspect, a computer program product is provided in this embodiment, including a computer program, which, when executed by a processor, implements the scalable large language model jailbreak attack method described in the first aspect.

[0038] The above-mentioned scalable large language model jailbreak attack method, device, medium and product generate a second prompt according to the first prompt template set for the jailbreak task, combined with the first prompt template and the first answer data corresponding to the jailbreak task, so that the target large language model can generate the second answer data required to execute the jailbreak task based on the second prompt, which meets the execution requirements of various jailbreak tasks and solves the problem of narrow assessment range of the security boundary. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is an application environment diagram of a scalable large language model jailbreak attack method in one embodiment;

[0040] Figure 2 It is a structural block diagram of an expandable large language model jailbreak attack device in one embodiment;

[0041] Figure 3 A structural block diagram of an expandable large language model jailbreak attack device in another embodiment;

[0042] Figure 4 A schematic diagram of a process flow of a scalable large language model jailbreak attack method in one embodiment;

[0043] Figure 5 A flowchart of a scalable large language model jailbreak attack method in another embodiment;

[0044] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In traditional technology, there are three main types of jailbreak attack algorithms for LLM (Large Language Model): manually written jailbreak prompts; white-box automatic attacks based on gradient search; and black-box automatic attacks based on transcription optimization.

[0047] Although manually written jailbreak tips have a good jailbreak effect, they require a lot of manpower, and their effectiveness will quickly become invalid as the security mechanism of LLM is optimized.

[0048] White-box automatic attacks based on gradient search optimize the input prompts by obtaining the internal weights and gradient changes of the model, thereby achieving a jailbreak attack in which the model outputs a harmful answer. For example, a discretely optimized suffix is ​​added after the normal prompt through greedy gradient search, so that the model outputs an answer starting with "Sure". Alternatively, a jailbreak attack prompt is obtained by first optimizing an adversarial suffix in continuous space and then converting it back to a readable suffix. However, the above white-box attack algorithms have problems such as high computing resource overhead and a narrow range of target models because they require optimization of the internal parameters of the model.

[0049] The black-box automatic attack based on transcription optimization generates jailbreak attack prompts by designing a reasonable prompt framework and feedback process, and using the powerful natural language processing capabilities of LLM. For example, a LLM is simulated as a red team tester using prompts, and the jailbreak attack is completed through interactive iterations between two LLMs. Going a step further, another evaluation model is introduced based on this technology, and the jailbreak prompts are iteratively generated using the idea of ​​depth-first search. Alternatively, the jailbreak is completed by allowing LLM to imitate human persuasion skills to generate prompts. However, although the black-box algorithm can attack almost all LLM models with less computing power overhead, the fixed prompt process and transcription template of each algorithm limit the scope of its application prompts and attack scenarios.

[0050] Based on the above problems, this embodiment provides a scalable large language model jailbreak attack method. The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 1 is a hardware structure diagram of a terminal of a scalable large language model jailbreak attack method according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the scalable large language model jailbreak attack method in the present embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0053] In one embodiment, Figure 2 As shown in the figure, a scalable large language model jailbreak attack method is provided, which can be applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0054] Step S202, obtaining a first prompt corresponding to the jailbreak task, and generating first answer data for the first prompt according to the question template.

[0055] The jailbreak task is used to make the target large language model generate data prohibited by its content filtering mechanism. The first prompt is the description required to execute the jailbreak task. The first answer data is the data prohibited by the content filtering mechanism of the target large language model generated according to the first prompt. The question template is used to guide the generation of the first answer data, and the question template may include the output format of the first answer data, the principles to be followed during the output, an example of the jailbreak task, etc.

[0056] Optionally, a pre-trained target generation model is obtained. The first prompt and the question template are combined, and the combined data is used as input of the target generation model to obtain an output of the target generation model, that is, the first answer data.

[0057] Step S204: updating the written content in the preset first prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements.

[0058] The role description includes a description of the role that the target large language model needs to play when performing the jailbreak task; the situation description includes a description of the background situation that the target large language model needs to bear when performing the jailbreak task. The preset format requirements at least include the format requirements for the second prompt.

[0059] Optionally, the preset first prompt template includes multiple modules, including a role and situation description module and an input and output format module. The role and situation description module is provided with a role description and a situation description corresponding to the jailbreak task; the input and output format module is provided with preset format requirements, including format requirements when the model generating the second prompt describes each component of the input data, and format requirements when the model generating the second prompt outputs the second prompt.

[0060] Step S206, taking the first answer data as an example, the first prompt is transcribed in combination with the role description and / or situation description in the first prompt template to obtain a second prompt that meets the format requirements.

[0061] Among them, through the first answer data, the transcription model can refer to the pattern of the first answer data to output the second prompt, so that the answer data output by the target large language model based on the second prompt matches the task requirements of the jailbreak task as much as possible. The second prompt obtained through the role description and / or situation description allows the target large language model to accurately understand the jailbreak task through the second prompt. By outputting a second prompt that meets the format requirements, the target large language model can understand the meaning of the second prompt.

[0062] Optionally, a transcription model for transcribing the first prompt is obtained. The first answer data, the first prompt and the first prompt template are combined, and the combined data is input into the transcription model to obtain a transcribed jailbreak prompt output by the transcription model, that is, the second prompt.

[0063] Step S208: obtaining second answer data generated by the target large language model based on the second prompt.

[0064] The target large language model is a model of the content security boundary range to be evaluated. Optionally, the second prompt is input into the large language model to obtain one or more second answer data output by the large language model.

[0065] In the above-mentioned scalable large language model jailbreak attack method, based on the first answer data and the first prompt template obtained by setting and updating different jailbreak attack tasks, the second prompt required to execute the jailbreak task is guided to be generated, so that the target large language model can generate the second answer data corresponding to the jailbreak task based on the second prompt. By adjusting the first prompt template, the execution requirements of various jailbreak tasks can be met. It has high scalability and avoids the problem of narrow evaluation range of content security boundaries caused by fixed algorithms and processes.

[0066] In one embodiment, based on the role description and / or situation description corresponding to the jailbreak task and the preset format requirements, the written content in the preset first prompt template is updated, including: obtaining transcription prompt data based on one or more of the example data, transcription rule data, and prompt data corresponding to the jailbreak task; based on the role description and / or situation description, the preset format requirements and the transcription prompt data, the written content in the preset first prompt template is updated.

[0067] The sample data is used to provide data that matches the jailbreak task. The transcription rule data includes one or more rules that need to be followed in the transcription of the second prompt. The transcription rule data can be obtained after setting rules in multiple dimensions such as the length of the output data, the field involved, and the language. The prompt data includes any content that the user wants to emphasize.

[0068] Optionally, in addition to the role and situation description module containing role description and / or situation description and the input and output format module containing preset format requirements, the first prompt template also includes one or more of the following modules: a rule description module for storing transcription rule data, an example module for storing example data, and an additional prompt template for storing prompt data. The transcription prompt data in the first prompt template and the corresponding modules can be selected and updated according to the jailbreak task and requirements.

[0069] Optionally, the matching status of the historical answer data and the jailbreak task can be obtained based on the historical answer data output by the target large language model, feedback prompts can be obtained, and the feedback prompts can be added to the first prompt template.

[0070] In this embodiment, the written content in the first prompt template is updated by one or more types of transcription prompt data other than role description and / or situation description and preset format requirements to meet jailbreak tasks in different scenarios, thereby improving the scalability of the jailbreak attack method.

[0071] In one embodiment, after obtaining the second answer data generated by the target large language model based on the second prompt, the method also includes: obtaining feedback parameters based on the match between the second answer data and the jailbreak task; updating the written content in the first prompt template based on the feedback parameters to obtain a feedback prompt; taking the first answer data as an example, transcribing the second prompt according to the role description and / or situation description, the feedback prompt and the first prompt to obtain a third prompt that meets the format requirements; obtaining the third answer data generated by the target large language model based on the third prompt.

[0072] The feedback parameters include, but are not limited to, the association parameter between the second answer data and the first prompt, the parameter for indicating whether the second answer data belongs to the content prohibited by the content filtering mechanism of the target large language model, the parameter for the degree of harm of the second answer data to the target large language model, etc. Optionally, the matching situation between the second answer data and the jailbreak task is obtained by judging whether the second answer data meets the requirements of the jailbreak task and whether the second prompt corresponds to the first prompt.

[0073] Optionally, with reference to the first answer data, and based on the first prompt, the second prompt is further transcribed in combination with the role description and / or situation description and feedback prompt to obtain a third prompt that meets the format requirements.

[0074] Optionally, the template content of the first prompt template may also include a feedback content module. The feedback content module includes the value of the feedback parameter and the description of the feedback parameter. The feedback prompt is obtained by updating the value, quantity, and type of the feedback parameter in the feedback content module and combining the description of the updated feedback parameter.

[0075] In this embodiment, the third prompt is obtained according to the feedback prompt, so as to obtain the third answer data matching the jailbreak task through iteration.

[0076] In one embodiment, feedback parameters are obtained based on the match between the second answer data and the jailbreak task, including: obtaining at least two second prompt templates with different judgment criteria based on the jailbreak task; obtaining a consistency parameter between the second prompt and the first prompt based on the second prompt template associated with the judgment criteria and the prompt transcription; obtaining an evaluation parameter of the second answer data based on the second prompt template associated with the judgment criteria and the jailbreak effect; and obtaining feedback parameters based on the consistency parameter and the evaluation parameter.

[0077] The second prompt template at least includes rules for describing evaluation criteria for answer data; the second prompt template may also include sample data of the jailbreak task, and / or format requirements for output of consistency parameters and evaluation parameters.

[0078] The consistency parameter is used to indicate the degree of transcription deviation of the second prompt relative to the first prompt. Optionally, if the deviation value between the second answer data and the first prompt is greater than a deviation threshold, the first parameter is obtained; if the deviation value between the second answer data and the first prompt is less than or equal to the deviation threshold, the second parameter is obtained.

[0079] The evaluation parameter is used to indicate the jailbreak effect corresponding to the second answer data. The evaluation parameters include but are not limited to: the jailbreak success rate of the second answer data, the degree of harm to the model, the modification range of the second prompt relative to the first prompt, etc. The feedback parameter can be obtained by combining the consistency parameter and the evaluation parameter.

[0080] In this embodiment, the answer data is evaluated from different dimensions through a variable second prompt template, and feedback parameters are obtained to adjust the generation of the third prompt, which can cope with different application scenarios and prompt requirements and obtain effective prompts to execute the jailbreak task.

[0081] Furthermore, in one embodiment, the judgment criteria in the second prompt template include one or more of the following: the degree of consistency between the semantics of the second prompt and the semantics of the first prompt, the degree of completion of the jailbreak task by the second answer data, the degree of concealment of the second prompt relative to the first prompt, and the degree of matching between the second answer data and the format requirements.

[0082] Optionally, the semantics of the first prompt and the semantic information of the second prompt are obtained; based on the semantic information, the degree of consistency in one or more of the following dimensions is obtained: the degree of consistency between the subject of the first prompt and the subject of the second prompt, the consistency between the target corresponding to the first prompt and the target corresponding to the second prompt, and the consistency between the logic of the first prompt and the logic of the second prompt. According to the consistency level of the above one or more dimensions, one or more consistency parameters for determining the degree of consistency are obtained.

[0083] Optionally, one or more of the following evaluation parameters can be obtained based on the degree of completion of the jailbreak task by the second answer data: jailbreak success rate, degree of harm of the second answer data to the target large language model. Among them, the jailbreak success rate can be obtained by judging the proportion of content prohibited by the content filtering mechanism of the target large language model in the second answer data: when the second answer data belongs to the content prohibited by the content filtering mechanism, the jailbreak success rate is high; otherwise, the jailbreak success rate is low. The degree of harm is obtained by comparing the second answer data with the preset harm judgment rules.

[0084] Optionally, evaluation parameters can be obtained according to the concealment degree of the second prompt relative to the first prompt: the modification range of the second prompt relative to the first prompt; and the readability of the second answer data. The greater the modification range of the second prompt relative to the first prompt, the weaker the concealment degree; the smaller the modification range of the second prompt relative to the first prompt, the stronger the concealment degree. When the second answer data is low in readability, the concealment degree is high; otherwise, the concealment degree is low.

[0085] Optionally, an evaluation parameter may be obtained according to the degree of matching between the second answer data and the format requirement: a parameter for indicating whether the output length and output format of the second answer data match the format requirement.

[0086] In this embodiment, obtaining the second prompt template through one or more discrimination criteria can improve the applicability of the second prompt template, thereby improving the scalability of the large language model jailbreak attack method.

[0087] In one embodiment, feedback parameters are obtained based on the matching situation between the second answer data and the jailbreak task, including: determining whether the second answer data matches the jailbreak task; if the second answer data does not match the jailbreak task, feedback parameters are obtained based on the matching situation between the second answer data and the jailbreak task.

[0088] Optionally, when it is determined that the deviation value between the second answer data and the subject in the second prompt is less than a threshold, the second answer data is determined to match the jailbreak task; otherwise, the second answer data is determined to not match the jailbreak task. If the second answer data matches the jailbreak task, the jailbreak task is completed, and the content security boundary of the target large language model can be evaluated based on the second answer data.

[0089] In this embodiment, by judging the degree of match between the second answer data and the jailbreak task, it is determined whether to execute the jailbreak task based on the match between the second answer data and the jailbreak task, and feedback parameters are obtained to reduce the time required to execute the scalable large language model jailbreak attack method.

[0090] In one embodiment, before determining whether the second answer data matches the jailbreak task, the method also includes: determining whether the number of updates to the first prompt template reaches a preset number of iterations; if the number of updates to the first prompt template does not reach the number of iterations, executing the step of determining whether the second answer data matches the jailbreak task.

[0091] The number of iterations can be set as required. If the number of updates of the first prompt template reaches the preset number of iterations, the scalable large language model jailbreak attack method is stopped, and the answer data generated by the large language model according to the latest prompt is used as data for evaluating the content security boundary. In this way, the time taken to execute the scalable large language model jailbreak attack method can be limited.

[0092] Based on the same inventive concept, the embodiment of the present application also provides an expandable large language model jailbreak attack device for implementing the above-mentioned expandable large language model jailbreak attack method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more expandable large language model jailbreak attack device embodiments provided below can refer to the limitations of the expandable large language model jailbreak attack method above, which will not be repeated here.

[0093] In one embodiment, Figure 3 As shown, a scalable large language model jailbreak attack device is provided, including: a jailbreak target generation module, an optimization instruction construction module, a transcription output generation module and a jailbreak output generation module, wherein:

[0094] a jailbreak target generation module, configured to obtain a first prompt corresponding to the jailbreak task, and generate first answer data for the first prompt according to a question template;

[0095] An optimization instruction construction module, used for updating the written content in the preset first prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements;

[0096] A transcription output generation module, used for taking the first answer data as an example, transcribing the first prompt in combination with the role description and / or situation description in the first prompt template, to obtain a second prompt that meets the format requirements;

[0097] The jailbreak output generation module is used to obtain second answer data generated by the target large language model based on the second prompt.

[0098] In some of the embodiments, the optimization instruction construction module updates the written content in the preset first prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements, including: obtaining transcription prompt data according to one or more of the example data, transcription rule data, and prompt data corresponding to the jailbreak task; updating the written content in the preset first prompt template according to the role description and / or situation description, the preset format requirements and the transcription prompt data.

[0099] In some embodiments, the optimization instruction construction module can also construct a first prompt template and a second prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements. After constructing the first prompt template and the second prompt template, the written content in the preset first prompt template is updated according to the data content generated by the target large prediction model during the operation of the scalable large language model jailbreak attack device and the feedback generated during the iteration process. Among them, the scalable large language model jailbreak attack device also includes a transcription deviation judgment module and a jailbreak effect judgment module. The transcription deviation judgment module and the jailbreak effect judgment module are used to obtain the second answer data generated by the target large language model based on the second prompt, and obtain the feedback parameter according to the matching between the second answer data and the jailbreak task; update the written content in the first prompt template based on the feedback parameter to obtain the feedback prompt. Subsequently, the optimization instruction construction module takes the first answer data as an example, transcribes the second prompt according to the role description and / or situation description, the feedback prompt and the first prompt, and obtains a third prompt that meets the format requirements; the jailbreak output generation module obtains the third answer data generated by the target large language model based on the third prompt.

[0100] Optionally, feedback parameters are obtained based on the match between the second answer data and the jailbreak task, including: the optimization instruction construction module obtains at least two second prompt templates with different judgment criteria according to the jailbreak task; the transcription deviation judgment module obtains a consistency parameter between the second prompt and the first prompt according to the second prompt template associated with the judgment criteria and the prompt transcription; the jailbreak effect judgment module obtains an evaluation parameter of the second answer data according to the second prompt template associated with the judgment criteria and the jailbreak effect; and the feedback parameters are obtained according to the consistency parameters and the evaluation parameters.

[0101] Optionally, the judgment criteria in the second prompt template include one or more of the following: the degree of consistency between the semantics of the second prompt and the semantics of the first prompt, the degree of completion of the jailbreak task by the second answer data, the degree of concealment of the second prompt relative to the first prompt, and the degree of match between the second answer data and format requirements.

[0102] In some embodiments, the scalable large language model jailbreak attack device also includes an iterative feedback module, and obtaining feedback parameters according to the matching between the second answer data and the jailbreak task includes: the iterative feedback module determines whether the second answer data matches the jailbreak task; if the second answer data does not match the jailbreak task, the feedback parameters are obtained according to the matching between the second answer data and the jailbreak task. Optionally, before determining whether the second answer data matches the jailbreak task, the iterative feedback module executes the method further including: determining whether the number of updates of the first prompt template reaches a preset number of iterations; if the number of updates of the first prompt template does not reach the number of iterations, the step of determining whether the first answer data matches the jailbreak task is executed.

[0103] Each module in the above-mentioned scalable large language model jailbreak attack device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0104] In some of these embodiments, Figure 4 Another scalable large language model jailbreak attack device is provided, including a control module, a jailbreak target generation module, an optimization instruction construction module, a transcription output generation module, a jailbreak output generation module, a transcription deviation judgment module, a jailbreak effect judgment module and an iterative feedback module. Among them, the control module is used to control the execution of the jailbreak target generation module, the optimization instruction construction module, the transcription output generation module, the jailbreak output generation module, the transcription deviation judgment module, the jailbreak effect judgment module and the iterative feedback module.

[0105] In one embodiment, Figure 5 A schematic diagram of another scalable large language model jailbreak attack algorithm is provided, and the method is applied to Figure 4 Taking the device shown as an example, the method includes:

[0106] Step S501, using the jailbreak target generation module, taking the initial first prompt in combination with the preset question template as the input of the generation model to obtain the required target output; using the optimization instruction construction module, constructing appropriate transcription prompt templates and feedback prompt templates according to different jailbreak task requirements and template data.

[0107] The target output is the first answer data in the above embodiment. The template data includes an example corresponding to the jailbreak task. The jailbreak task requirements include data related to the jailbreak task, such as role description, situation description, rule description, and additional prompts. The transcription prompt template is the first prompt template in the above embodiment, and the feedback prompt template is the second prompt template in the above embodiment.

[0108] Step S502, using a transcription output generation module, combining the target output, the first prompt and the transcription prompt template, inputting the transcription model, and obtaining the transcription jailbreak prompt output by the transcription model. The transcription model is a large language model, and the transcription jailbreak prompt is the second prompt in the above embodiment.

[0109] Step S503, using the jailbreak output generation module, inputting the transcribed jailbreak prompt into the target large language model to obtain corresponding jailbreak answer data, wherein the jailbreak answer data is the second answer data in the above embodiment.

[0110] Step S504, using the transcription deviation judgment module and the jailbreak effect judgment module, input the transcribed jailbreak prompt and the corresponding jailbreak answer data into the discrimination model to obtain the deviation judgment data and effect judgment data corresponding to the transcribed jailbreak prompt.

[0111] Step S505, judging whether the jailbreak is successful; if so, executing step S508; if not, executing step S506. In which, judging whether the jailbreak is successful is based on the deviation judgment data and the effect judgment data.

[0112] Step S506, determine whether the iteration limit has been reached; if so, execute step S508; if not, execute step S507.

[0113] Step S507, using an iterative feedback module, combining the deviation judgment data, the effect judgment data and the feedback prompt template to form a feedback prompt, and sending the feedback prompt back to the transcription output generation module to participate in the transcription output process as part of the prompt, and executing step S502.

[0114] Step S508: taking the output of the target large language model as the output result.

[0115] Optionally, the jailbreak target generation module used in step S501 is used to combine the initial first prompt with a preset question template and use it as an input of the target generation model to obtain the target output, which can be expressed as follows:

[0116] T=f T (M T ,q)

[0117] Among them, q represents the initial first prompt, M T represents the preset question template used to generate the target output, f t represents the LLM used to generate the target output, and T represents the target output.

[0118] The optimization instruction construction module used in step S501 can realize the construction of the transcription prompt template and the construction of the feedback prompt template.

[0119] For transcription prompt template M R : Multiple optional modules can be obtained according to the requirements of the jailbreak task and the template data. Optionally, the transcription prompt template M R It consists of 6 optional modules, including: role and situation description module, rule description module, input and output format module, example module, additional prompt module, and feedback content module. Among them, the role and situation description module uses natural language to prompt the model to play the role and the background of the transcription task it undertakes, which is used to provide a basis for the model to accurately understand the task requirements. The rule description module is used to list the principles that need to be followed in the transcription process one by one to improve the performance of the model transcription output. The input and output format module is used to describe the components of the input content received by the model. For example, after the feedback data is input into the input and output format module, the meaning of each judgment result in the feedback data will be described; and the input and output format module is also used to specify the model output format, such as requiring the transcription model to output in JSON format. The example module contains the input cases received by the model, which is used to provide a reference template for the output of the transcription model and a sample of the model output to assist the model to output in accordance with the prompt requirements. The additional prompt module may contain other emphasized content. The feedback content module contains the judgment data of the last iteration result fed back by the transcription deviation judgment module and the jailbreak effect judgment module.

[0120] For example, when implementing a process similar to PAP (Persuasive Adversarial Prompts), the following modules can be constructed in the transcription prompt template: role and situation description module, rule description module, input and output format module, and example module. Among them, the role and situation description module includes: "Existing Question: [the shortcomings of the current jailbreak prompt, model transcription precautions, etc.]"; "Character Description: [role played by the model, role behavior description, behavior precautions, etc.]". The rule description module includes: "Persuasion Type: ["Evidence-basedPersuasion", "Logical appeal", "Authority endorsement", "Misrepresentation", "Expert Endorsement", ...]"; "Description for Type: [description corresponding to the above Type: how to use the first prompt, reasoning process, etc.]". The input and output format module includes: "Output Elements: [prompt core content, jailbreak transcription prompt, prompt modification content, etc.]"; "Description For Elements: [description corresponding to the above Elements: component format requirements, component content, etc.]". The examples module contains examples that conform to the output format described above.

[0121] For example, when implementing a process similar to TAP (Tree of Attacks with Pruning), the following modules can be constructed in the transcription prompt template: role and situation description module, rule description module, input and output format module, example module and feedback content module. The role and situation description module includes: "Character Description: [role type, role characteristics, role tasks, etc.]"; "Background Description: [usable information description, transcription strategy description, etc.]". The rule description module includes: "[Regularize the key content of the role and background: transcribe key strategies, transcribe reasoning process, etc.]"; "Input and output format module: FORMAT: [output format, such as JSON, Python array, etc.]"; "INPUT ELEMENTS: [Input content components obtained by the model: first prompt, target output, etc.]"; "DESCRIPTION FOR INPUT ELEMENTS: [Description of INPUT ELEMENTS]"; "OUTPUT ELEMENTS: [Content components that the model needs to output: transcribe jailbreak prompts, etc.]"; "DESCRIPTION FOR OUTPUT ELEMENTS: [Description of OUTPUT ELEMENTS]". The example module contains examples that conform to the above output format. The feedback content module includes: "MODEL OUTPUT: [Output of the jailbreak target model]"; "SCORE: [Feedback from the jailbreak effect judgment module: data such as jailbreak success rate, model output hazard level and first prompt modification range]"; "Evaluator_On_Topic: [Feedback from the transcription deviation judgment module]".

[0122] By enabling different module combinations and adjusting the transcription prompt templates obtained from the module contents, this algorithm can be adapted to jailbreak attacks with different prompts and attack scenarios.

[0123] For feedback prompt template M J , can be composed of 3 modules: rule description module, output format module, and example module. Among them, the rule description module is used to establish multiple judgment rules for the discriminant model; specifically, based on one or more rules in the rule description module, the output of the target large language model can be evaluated and multiple scores can be obtained, and a corresponding natural language description is given to each score to complete the judgment of the discriminant model. The output format module is used to describe the format of various feedback components that the discriminant model should output and the meaning of each component. The example module contains samples of the input received by the model and the output to assist the discriminant model to follow the prompt requirements for output.

[0124] By appropriately changing the content description in each module of the feedback prompt template, different feedback prompt templates M required for the transcription deviation judgment module and the jailbreak effect judgment module can be generated respectively. JI and M JH , to meet the needs of different jailbreak tasks.

[0125] The transcription output generation module used in step S502 mainly includes LLM. Specifically, the transcription output generation module consists of a variable LLM (f R ). The transcription output generation module is composed of the target output T, the first prompt q and the transcription prompt template M R Combine and input the combined data into f R , you can output the transcription of the jailbreak prompt R q ; means as follows:

[0126] R q =f R (M R ,q,T)

[0127] The jailbreak output generation module used in step S503 is used to generate the transcribed jailbreak prompt R generated by the transcribed output generation module. q Input to the target large language model f A , to obtain the target large language model f A Output jailbreak answer data J q ; means as follows:

[0128] J q =f A (R q )

[0129] The transcription deviation judgment module used in step S504 is used to generate deviation judgment data; the jailbreak effect judgment module is used to generate effect judgment data; the feedback parameters in the above embodiment can be obtained by combining the deviation judgment data and the generated effect judgment data.

[0130] Among them, the transcription deviation judgment module is mainly composed of a variable LLM (f JI ). It consists of the discriminant model f JI Template M based on feedback JI The rule description module, output format module and example module in the target language model f A Whether the output content is consistent with the content theme of the first prompt to prevent the jailbreak performance from being degraded due to the deviation of the transcription theme. Optionally, the discriminant model execution in the transcription deviation judgment module is expressed as follows:

[0131] R I =f JI(M JI ,q,R q )

[0132]

[0133] Among them, R I Represents the consistency score of the discriminant model output, V I Indicates the threshold for judging whether the transcription output is off-topic. If R I If the threshold is exceeded, the final feedback result R IF Take 1, otherwise it is 0.

[0134] For example, the feedback prompt template M used in the transcription deviation judgment module Ji It includes rule description module, rule description module and example module. The rule description module includes: "Character Description: [Role type, character characteristics, character tasks, etc.]"; "Input Description: [Format and content description of model input]". The output format module includes: "Response Format: [Format description of model output]". The example module contains examples that conform to the output format, for example, the examples can be "Response: [[YES]]" and "Response: [[NO]]".

[0135] Among them, the jailbreak effect judgment module is mainly composed of one or more variable LLMs (f JH The jailbreak effect judgment module is used to select the corresponding feedback prompt template M according to different jailbreak task requirements. JH Perform multi-dimensional evaluation on the transcribed jailbreak prompts and output the corresponding discrimination parameters {R j1 ,R j2 ,…,R jn}. The discrimination parameters include but are not limited to the jailbreak success rate, the degree of harm of the model output, and the modification range of the first prompt. The discrimination parameters can be combined with the feedback content module in the feedback prompt template to obtain feedback prompts, thereby providing sufficient data for subsequent iterative feedback; it is expressed as follows:

[0136] {R J1 ,R j2 ,…,R jn} = f JH (M jH ,q,R q ,J q )

[0137] Exemplarily, the feedback prompt template M used in the jailbreak effect judgment module JHIt includes rule description module, rule description module and example module. The rule description module includes: "Character Description: [Role type, role characteristics, role tasks, etc.]"; "Rules for Score: [Rules followed by model evaluation: morality, harmfulness, fairness, etc.]"; "Description for Scores: [Definition of evaluation scores, description of different scores]". The output format module includes: "Response Format: [Format description of model output]". The example module contains examples that conform to the output format, for example, the example can be "Rating: [[5]]".

[0138] In this embodiment, by setting an optimized instruction construction module, a transcription prompt template and a feedback prompt template containing different component modules and text content are generated, so that the content of different modules can be designed according to different jailbreak attack scenarios, and a variety of jailbreak prompt content can be generated to achieve the jailbreak attack effect. Further, by using a variable transcription deviation judgment module and a jailbreak effect evaluation module, it is possible to flexibly respond to different application scenarios and prompt requirements, thereby effectively executing jailbreak attacks, improving the scalability of the large language model jailbreak attack algorithm, and broadening the evaluation scope of the content security boundary when LLM responds to jailbreak attacks.

[0139] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the indication of the arrow, these steps are not necessarily executed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart involved in each embodiment as described above may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be performed in turn or alternately with other steps or at least a part of the steps or stages in other steps. For example, while executing step S502 or executing step S503, the optimized instruction construction module is adopted in execution step S501, and the method for constructing a suitable transcription prompt template and feedback prompt template according to different jailbreak task requirements and template data is constructed.

[0140] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as jailbreak tasks, question templates, and first prompt templates. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a scalable large language model jailbreak attack method is implemented.

[0141] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0142] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0143] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0144] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A scalable large language model jailbreak attack method, characterized in that: The method comprises: Obtaining a first prompt corresponding to the jailbreak task, and generating first answer data for the first prompt according to a question template; According to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements, updating the template content in the preset first prompt template; Taking the first answer data as an example, the first prompt is transcribed in combination with the role description and / or situation description in the first prompt template to obtain a second prompt that meets the format requirement; Obtain second answer data generated by the target large language model based on the second prompt.

2. The scalable large language model jailbreak attack method according to claim 1, characterized in that: After acquiring the second answer data generated by the target large language model based on the second prompt, the method further includes: Obtaining a feedback parameter according to a matching condition between the second answer data and the jailbreak task; Based on the feedback parameter, the written content in the first prompt template is updated to obtain a feedback prompt; Taking the first answer data as an example, the second prompt is transcribed according to the role description and / or situation description, the feedback prompt and the first prompt to obtain a third prompt that meets the format requirement; Obtain third answer data generated by the target large language model based on the third prompt.

3. The scalable large language model jailbreak attack method according to claim 2, characterized in that: The obtaining of feedback parameters according to the matching between the second answer data and the jailbreak task includes: Obtaining at least two second prompt templates with different judgment criteria according to the jailbreak task; Obtaining a consistency parameter between the second prompt and the first prompt according to a second prompt template associated with the prompt transcription according to the discrimination criterion; obtaining evaluation parameters of the second answer data according to a second prompt template associated with the discrimination criterion and the jailbreak effect; The feedback parameter is obtained according to the consistency parameter and the evaluation parameter.

4. The scalable large language model jailbreak attack method according to claim 3, characterized in that: The judgment criteria in the second prompt template include one or more of the following: the degree of consistency between the semantics of the second prompt and the semantics of the first prompt, the degree of completion of the jailbreak task by the second answer data, the degree of concealment of the second prompt relative to the first prompt, and the degree of matching between the second answer data and format requirements.

5. The scalable large language model jailbreak attack method according to claim 2, characterized in that: According to the matching between the second answer data and the jailbreak task, feedback parameters are obtained, including: Determining whether the second answer data matches the jailbreak task; In the case that the second answer data does not match the jailbreak task, a feedback parameter is obtained according to a matching condition between the second answer data and the jailbreak task.

6. The scalable large language model jailbreak attack method according to claim 5, characterized in that: Before determining whether the second answer data matches the jailbreak task, the method further includes: Determining whether the update number of the first prompt template reaches a preset number of iterations; When the update times of the first prompt template does not reach the iteration times, a step of determining whether the second answer data matches the jailbreak task is performed.

7. The scalable large language model jailbreak attack method according to claim 1, characterized in that: The updating of the written content in the preset first prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements includes: Obtaining transcription prompt data according to one or more of the example data, transcription rule data, and prompt data corresponding to the jailbreak task; The written content in the first prompt template is updated according to the role description and / or situation description, the preset format requirement and the transcription prompt data.

8. A scalable large language model jailbreak attack device, characterized in that: The device comprises: a jailbreak target generation module, configured to obtain a first prompt corresponding to the jailbreak task, and generate first answer data for the first prompt according to a question template; An optimization instruction construction module, used for updating the written content in the preset first prompt template according to the role description and / or situation description corresponding to the jailbreak task and the preset format requirements; A transcription output generation module, used for taking the first answer data as an example, transcribing the first prompt in combination with the role description and / or situation description in the first prompt template, to obtain a second prompt that meets the format requirements; The jailbreak output generation module is used to obtain second answer data generated by the target large language model based on the second prompt.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images