Large language model jailbreak method based on danger decoding

Abstract

The invention relates to a big language model jailbreak method based on danger decoding, which is characterized in that token generation of a target model is guided through a decoding path of a danger model based on a big language model, after an updated token space is obtained through recombination of the token space, decoding is carried out according to a determined decoding mode, and then a token can be generated. The standard for judging whether the prison break succeeds is that whether the generated reply content effectively replies the malicious prompt word or not. According to the method, a small model is finely adjusted to become a danger model, and the token space of a target model is updated by utilizing the characteristic that the probability of a positive reply token in the token space of the danger model is relatively high, so that the target model finally performs positive reply on malicious prompt words. According to the method, the time consumption is reduced, the success rate of jailbreak attacks is improved, the open source model can be well subjected to jailbreak, and compared with the prior art, the universality is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of jailbreak attack targeting large model security vulnerabilities, and in particular to a large language model jailbreak method based on dangerous decoding. Background Art

[0002] Large Language Models (LLMs) such as ChatGPT, Llama2, Vicuna, and Gemini have experienced remarkable progress. Despite these advances, they have encountered significant challenges in terms of security. The urgent need for strong security measures has been highlighted in several reports. There has been a lot of work to align the behavior of large language models with human values ​​to ensure that their output content is helpful and non-harmful. Despite advances in alignment techniques, the output of the models is still vulnerable to hostile inputs. Recent research has revealed a significant threat known as the "jailbreak attack" that can successfully bypass existing security alignments.

[0003] At present, although there are some attack methods for large language models, such as adversarial sample attacks, model stealing, data poisoning and prompt optimization, these methods have the following limitations: adversarial sample attacks often require a complex optimization process, which is usually very time-consuming, and the effect may be affected by the model structure and training data; model stealing and data poisoning attacks require a certain degree of control over the model training process, which may be difficult to achieve in practical applications; prompt optimization methods usually require calling multiple third-party models, which seriously wastes computing power resources; existing attack methods may not be able to effectively target all types of large language models. Summary of the invention

[0004] The purpose of the present invention is to provide a large language model jailbreak method based on dangerous decoding to solve the problems encountered in the above-mentioned background technology.

[0005] To achieve the above object, the technical solution of the present invention is as follows: A large language model jailbreak method based on dangerous decoding, the method comprising the following steps: Step 1, selection and fine-tuning of dangerous models: The large language model jailbreak method based on dangerous decoding guides the token generation of the target model through the decoding path of a dangerous model. Here, a smaller model is selected for fine-tuning, and the fine-tuning dataset format is in the form of harmful prompt words and response pairs.

[0006] Finally, this smaller model is used to jailbreak the target model, which consumes less computing power and time. The fine-tuning steps are relatively simple, and only 100 data sets are needed to fine-tune an effective dangerous model.

[0007] Step 2, decoding generation stage of target model and danger model: copy the malicious prompt word into two copies, and ask questions to the target model and danger model at the same time. The two models are decoded according to the same decoding method (such as decoding based on the greedy algorithm), and finally two token spaces are obtained. The token space contains information about the possible generated tokens and the corresponding probabilities.

[0008] Copying the malicious prompt words into two copies and inputting them into the target model and the danger model at the same time consumes less time.

[0009] Step 3, reorganization of token space: At this stage, the token space of the target model needs to be reorganized. First, select the common tokens in the two token spaces, and then calculate the corresponding probabilities of these tokens according to the designed formula to obtain the updated probabilities. Finally, sort the tokens from high to low according to the probability. The reorganized token space has a great tendency to actively respond to malicious prompt words and generate harmful content.

[0010] Through the designed formula, the probability of tokens that tend to refuse to answer in the target model token space is reduced, and the probability of tokens that tend to respond affirmatively is increased, which makes it more likely to respond affirmatively to the malicious prompt word during the final decoding, thereby successfully jailbreaking.

[0011] Step 4, the final generation stage of the target model: After the token space is reorganized to obtain the updated token space, a token can be generated by decoding according to the determined decoding method. This solution does not require steps 2 and 3 to be performed at each step of token generation. There is a hyperparameter m here that indicates that the first m tokens are decoded dangerously, and only the target model needs to be decoded normally afterwards, which can reduce the waste of resources.

[0012] The reason why the hyperparameter m is used here instead of performing dangerous decoding in the entire generation process is that the time consumed by performing dangerous decoding in the entire process is greater than that of using the dangerous decoding strategy for the first m tokens. Moreover, when m is greater than 40, the jailbreak success rate is not much different from that of performing dangerous decoding in the entire process.

[0013] Step 5, evaluation phase: Based on the generated tokens, the large language model is input to determine whether the jailbreak is successful based on whether the generated reply content effectively replies to the malicious prompt words. Here, GPT4 is used to score the harmfulness of the reply content and formulate relevant evaluation criteria.

[0014] Compared with the prior art, the present invention has the following beneficial effects: By using the dangerous model to reorganize the target model decoding token space, an effective jailbreak attack scheme is proposed for the existing open source large model; in the dangerous model selection stage, a smaller model is used for fine-tuning to achieve the effect of small wins and big losses, reducing the hardware conditions for jailbreaking; in the decoding generation stage of the target model and the dangerous model, the malicious prompt words are copied into two copies and input into the two models at the same time, which effectively reduces the jailbreak time; in the token space reorganization stage, considering that the whole process of dangerous decoding has little effect on the success rate of jailbreaking, the first m tokens are used for dangerous decoding, which effectively reduces the time to generate the final token; in the evaluation stage, considering that the model will have a "regret phenomenon", that is, it generates affirmative response content when dangerous decoding, and generates rejection response content when normal decoding, instead of using a simple keyword detection scheme, the overall response content is scored for harmfulness, which effectively increases the evaluation accuracy.

[0015] In order to solve the problems of low attack success rate, high time consumption and low versatility of existing jailbreak attack methods, the present invention proposes a large language model jailbreak method based on dangerous decoding. By fine-tuning a small model into a dangerous model, the token space of the target model is updated by using the characteristic that the probability of a token with a positive reply in the token space of the dangerous model is high, so that it can finally make a positive reply to the malicious prompt word. This method reduces time consumption, improves the success rate of jailbreak attacks, can jailbreak open source models well, and is more versatile than the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the accompanying drawings, the same reference numerals are used to refer to the same components. Among them: Figure 1 This is a flowchart of the jailbreak attack based on dangerous decoding of the present invention; Figure 2 This is an example diagram of the jailbreak attack principle based on dangerous decoding provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the technical means, creative features, objectives and effects of the present invention easy to understand, the present invention is now further described in detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the relevant components of the present invention.

[0018] According to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific implementation modes and the accompanying drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the entirety of the present invention or as a limitation or restriction to the technical solution of the present invention.

[0019] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0020] Embodiment 1, as Figure 1 As shown, a large language model jailbreak method based on dangerous decoding, the method specifically includes the following steps: Step 1: Selection and fine-tuning of the danger model. A smaller open source model is selected for fine-tuning. The dataset used is 100 queries randomly selected from the public Advbench dataset. The positive reply content generated by the GPT Fuzzer jailbreak method is used as the response. The fine-tuning dataset D is shown below, where N is 100. The fine-tuning scheme uses common supervised fine-tuning to obtain an effective danger model.

[0021] D=

[0022] Step 2, decoding generation stage of target model and dangerous model: copy the malicious prompt word into two copies, and ask questions to the target model and dangerous model at the same time. The two models are decoded in the same decoding method (such as greedy decoding). Here, the same decoder as the model is used for decoding, and finally two token spaces can be obtained. The token space contains information about possible generated tokens and corresponding probabilities. Here, each model can be set to only display the top ten token information in probability ranking, because the tokens that the model may generate are basically generated from the top ten tokens in probability ranking.

[0023] Step 3, reorganization of token space. Some parameters need to be set before token reorganization. When updating the token space of the target model, the size of the token space l needs to be set. The size of the token space l is related to the attack success rate. Here l is selected as 5. The token we want to operate is the token that exists in the token space of both models. Our danger decoding scheme mainly uses the probability difference between the danger model and the target model, multiplying the probability difference by a hyperparameter α, and then adding the result to the probability of the corresponding token generated by the danger model. The formula is as follows. α is generally taken as 2.

[0024] P = +α( ) Step 4, the final generation stage of the target model, uses the first m tokens to perform dangerous decoding using the above formula, and the default method is to use a greedy method for decoding. After the mth token, only the target model is decoded normally until the end. The value of m has a great impact on the success rate of jailbreak. It is generally 40 or 60, because after decoding the first 40 or 60 tokens, the reply content has generated enough harmful content, and the probability of subsequent "repentance" and refusing to reply is extremely low, which has a low impact on the final success rate of jailbreak attacks, so the whole process is not used for dangerous decoding.

[0025] Step 5, in the final evaluation stage, the token produced is input into the large language model, and a harm assessment scheme is adopted to determine whether the jailbreak is successful. The prompt words are designed by yourself to ask GPT4 questions, and the evaluation criteria are given. Let GPT4 output the harm score, with a full score of 5 points, and the harm from 1 to 5 gradually increases. A harm value of 5 points is regarded as a successful jailbreak, that is, an effective response to the corresponding prompt. This method is relatively general, and the specific prompt word design will not be shown here. The overall schematic diagram is as follows Figure 2 shown.

[0026] Example 2, based on Example 1, as a preferred solution, performs the following operation process: In step 1, select Llama2-7b for fine-tuning, adopt the supervised fine-tuning training scheme, the data set is the above 100 data, execute 10 epochs, and then get the corresponding adapter file. When calling the danger model, load the original model and adapter file.

[0027] In step 2, the decoding generation stage of the target model and the dangerous model: the target model used here is Llama2-70b. After loading the target model, copy the malicious prompt words you want to ask twice and input them into the two models at the same time. The two models use the greedy decoding method by default to obtain the decoding space of the two models. Here, only the information of the top ten tokens in probability ranking is displayed.

[0028] In step 3, the token space is reorganized: the number of identical tokens is set to 5, which is the size of the updated token space l, and α is set to 2. After setting the parameters, the first m tokens are decoded according to the scheme described above.

[0029] In step 4, the final generation stage of the target model: m is set to 40, and after the 40th token, only the target model is decoded normally to generate subsequent content until the end.

[0030] In step 5, select GPT4 as the evaluation model. You can also select other large language models. There are many public prompt word template designs. You can choose them yourself, or you can design your own prompt words.

[0031] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large language model jailbreak method based on dangerous decoding, characterized in that: The method comprises the following steps: Step 1: Selection and fine-tuning of the danger model: Based on the large language model, the decoding path of a danger model is used to guide the token generation of the target model; Step 2, the final generation stage of the target model: after the token space is reorganized to obtain the updated token space, a token can be generated by decoding according to the determined decoding method; Step 3, evaluation phase: Based on the generated token input into the large language model, the criterion for judging whether the jailbreak is successful is whether the generated reply content effectively replies to the malicious prompt words, and the harmfulness of the reply content is scored, and relevant evaluation criteria are formulated.

2. According to claim 1, a large language model jailbreak method based on dangerous decoding is characterized in that: In step 1, a smaller model is selected for fine-tuning, and the fine-tuning dataset is in the form of harmful prompt word and response pairs.

3. The large language model jailbreak method based on dangerous decoding according to claim 1 is characterized by: Between step 1 and step 2, the following steps are also performed: Step 1.1, decoding generation phase of target model and danger model: copy the malicious prompt word into two copies, and ask questions to the target model and danger model at the same time. The two models decode in the same way, and finally obtain two token spaces, which contain information about possible generated tokens and corresponding probabilities; Step 1.2, reorganization of token space: reorganize the token space of the target model, first select the common tokens in the two token spaces, then calculate the corresponding probabilities of these tokens according to the designed formula to obtain the updated probabilities, and finally sort the tokens from high to low according to the probability.

4. The large language model jailbreak method based on dangerous decoding according to claim 1 is characterized in that: In step 2, a hyperparameter m is set to indicate that the first m tokens are decoded dangerously, and only the target model is decoded normally afterwards.

5. The large language model jailbreak method based on dangerous decoding according to claim 4 is characterized in that: m≤60。 6. The large language model jailbreak method based on dangerous decoding according to claim 4 is characterized in that: In step 1.2, the probability difference between the danger model and the target model is used to multiply the probability difference by a hyperparameter α, and then the result is added to the probability of the corresponding token generated by the danger model. The formula is as follows. α is generally taken as 2, and the designed formula is: P = +α（ ）。

Images