Attack defense method, system and equipment based on large language model and storage medium

Abstract

The invention discloses an attack defense method, system and device based on a large language model, and a storage medium. The method comprises the following steps: constructing a target attack instruction and a target defense instruction based on an original task instruction and a target task instruction; constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction; calling the large language model to output model response content according to the model input instruction set; and determining a defense instruction of the large language model according to the model response content so as to defend various attack instructions. By constructing the target attack instruction and the target defense instruction, the attack and defense process is simulated, the robustness of the large language model in the face of various attacks can be improved, and the large language model is more stable and reliable in practical application.

Description

Technical Field

[0001] The present invention relates to the field of big data, and in particular to an attack defense method, system, device and storage medium based on a large language model. Background Art

[0002] Large Language Models (LLMs) are artificial intelligence models trained using large-scale text datasets, with the main goal of understanding and generating natural language text. With the booming development of large language model technology, its potential security risks and hidden dangers have become increasingly prominent, causing widespread concern and worry. When using large language models, specific prompts or instructions are usually input into the model to guide the model to generate corresponding outputs. By carefully designing these prompts, the model can be guided to complete tasks such as answering questions, writing articles, or writing code. However, this also brings a series of security issues. For example, attackers can use carefully designed prompts to induce the model to bypass the security audit mechanism, thereby achieving "jailbreak attacks" or "target hijacking attacks". These attacks may abuse the capabilities of the model, affect the actual user experience, and lead to the generation of inappropriate or harmful content.

[0003] Target hijacking attacks use carefully designed prompts to mislead the model into abandoning the user's original task instructions and executing the target task set by the attacker. This attack method can make the model produce specific outputs or behaviors to achieve the attacker's specific goals. Most of the current large model defense technologies are based on "prompt filtering" methods, which are mainly used to defend against potential sensitive content and are not used for target hijacking attacks. The rare "prompt enhancement" method mainly relies on manual writing, and the defense effect is limited. Summary of the invention

[0004] The main purpose of the present invention is to provide an attack and defense method, system, device and storage medium for a large language model. By constructing target attack instructions and target defense instructions and simulating the attack and defense process, it helps to improve the robustness of the large language model in the face of various attacks, making it more stable and reliable in practical applications.

[0005] In order to achieve the above objectives, the present application provides the following technical solutions:

[0006] According to a first aspect of an embodiment of the present application, an attack defense method based on a large language model is provided, the method comprising:

[0007] Constructing target attack instructions and target defense instructions based on original task instructions and target task instructions;

[0008] Constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction;

[0009] Calling the large language model to output model response content according to the model input instruction set;

[0010] The defense instructions of the large language model are determined according to the model response content to defend against various attack instructions.

[0011] Optionally, constructing a target attack instruction and a target defense instruction based on the original task instruction and the target task instruction includes:

[0012] Constructing an original task instruction response pair set according to the original task instruction, wherein the instruction response pair set includes a plurality of one-to-one corresponding instructions and responses;

[0013] Constructing a target task instruction response pair set according to the target task instruction;

[0014] Constructing a target attack instruction according to the original task instruction response pair set and the target task instruction response pair set, wherein the target attack instruction is used to make the large language model ignore the original task instruction response pair set and only execute the target task instruction response pair set;

[0015] A target defense instruction is constructed in the model token vocabulary of the large language model according to the original task instruction response pair set and the target task instruction response pair set, and the target defense instruction is used to defend against possible attacks.

[0016] Optionally, constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction comprises:

[0017] Sequentially traverse each pair of original task instructions and original task responses in the original task instruction response pair set, and sequentially traverse each pair of target task instructions and target task responses in the target task instruction response pair set;

[0018] At each step in the traversal process, the original task instruction, the target hijacking instruction, the target task instruction, and the target defense instruction are sequentially spliced ​​to obtain the input instruction of this step;

[0019] After the traversal process is completed, the input instructions of each step are combined into the model input instruction set.

[0020] Optionally, determining the defense instruction of the large language model according to the model response content includes:

[0021] Calculate the similarity between the model response content and the corresponding original task response, recorded as the first similarity;

[0022] Calculate the similarity between the model response content and the corresponding target task response, recorded as the second similarity;

[0023] Calculating a defense success rate based on the first similarity and the second similarity;

[0024] The target defense instruction whose defense success rate meets the set condition is determined as the defense instruction of the large language model.

[0025] Optionally, the method further comprises:

[0026] If the defense success rate does not meet the set condition, the target defense instruction is iteratively optimized until the defense success rate meets the set condition.

[0027] Optionally, iteratively optimizing the target defense instruction includes:

[0028] reconstructing the target defense instructions;

[0029] Construct a new model input instruction set according to the reconstructed target defense instruction, the original task instruction, the target task instruction, and the target hijacking instruction;

[0030] Calling the large language model to output corresponding model response content according to the new model input instruction set;

[0031] A new defense success rate is calculated according to the model response content corresponding to the new model input instruction set until the new defense success rate meets the set condition.

[0032] Optionally, the reconstructing the target defense instruction includes:

[0033] Reselecting a number of tokens from the model token vocabulary of the large language model to form a character string;

[0034] selecting a starting position in the target defense instruction;

[0035] The character string is used to replace the characters in the target defense instruction that have the same length as the character string starting from the starting position, to obtain a new target defense instruction after the replacement.

[0036] According to a second aspect of an embodiment of the present application, an attack defense system based on a large language model is provided, wherein the system comprises:

[0037] An instruction construction module, used for constructing a target attack instruction and a target defense instruction based on the original task instruction and the target task instruction;

[0038] An instruction splicing module, used for constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction;

[0039] A model application module, used for calling the large language model to output model response content according to the model input instruction set;

[0040] The defense instruction determination module is used to determine the defense instructions of the large language model according to the model response content to defend against various attack instructions.

[0041] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0042] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described in the first aspect above.

[0043] In summary, the embodiments of the present application provide an attack defense method, system, device, and storage medium for a large language model, by constructing a target attack instruction and a target defense instruction based on the original task instruction and the target task instruction; constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction, and the target defense instruction; calling the large language model to output a model response content according to the model input instruction set; determining the defense instruction of the large language model according to the model response content to defend against various attack instructions. By constructing target attack instructions and target defense instructions and simulating the process of attack and defense, it helps to improve the robustness of the large language model in the face of various attacks, making it more stable and reliable in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0045] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0046] Figure 1 A schematic diagram of a flow chart of a method for defending against attacks on a large language model provided in an embodiment of the present application;

[0047] Figure 2 A defense logic diagram for hijacking attacks provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of a large language model provided in an embodiment of the present application;

[0049] Figure 4 A block diagram of an attack defense system for a large language model provided in an embodiment of the present application;

[0050] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present application is shown;

[0051] Figure 6 A diagram showing a computer-readable storage medium provided by an embodiment of the present application.

[0052] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0055] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0056] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0058] Ensuring that the large model accurately responds to user instructions and guarantees the security compliance of the output content is crucial for its application in actual business scenarios. Defense strategies can be divided into two parts according to the location of the strategy: input-side defense and output-side defense. Input-side defense mainly includes two strategies: prompt filtering and prompt enhancement. Prompt filtering aims to filter prompt injections and potential sensitive content that may lead to attacks, while prompt enhancement is used to defend against prompt injection attacks such as target hijacking, prompt leakage, and jailbreaking. Prompt enhancement is a technology that aims to build more robust defensive prompts to enhance the ability of system prompts to resist prompt injection attacks. It uses the understanding ability of the large language model itself for "self-enhancement", and adds emphasis on the task content and user input content in the prompt words to form a more accurate system prompt for the task description, helping the large language model to better understand and complete the target task. The defense means involved in the present invention is a prompt enhancement type.

[0059] There are few studies on target hijacking attacks in the prior art, but more general instruction attacks, which usually make the model output sensitive content. These attacks are also optimized during the training phase. Due to its particularity, target hijacking attacks usually do not allow the model to execute sensitive or unsafe content. Instead, they hijack large models in a "normal" way so that they cannot work properly. The embodiments of the present application can fill this research gap and greatly reduce the risk of target hijacking attacks.

[0060] The embodiments of the present application are flexible, simple and easy to implement. Specifically, the required data can be easily collected, there is no need to update model parameters, and there is no need to worry about the model capabilities being affected. Compared with the defense method based on filtering and intercepting input and output, directly adding target hijacking defense instructions helps to improve the user experience of large language models, invalidate the attack from the root, and still have a good response to the user's original instructions, improving the user experience.

[0061] Figure 1 An attack defense method based on a large language model provided in an embodiment of the present application is shown, and the method includes:

[0062] Step 101: constructing a target attack instruction and a target defense instruction based on the original task instruction and the target task instruction;

[0063] Step 102: constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction;

[0064] Step 103: calling the large language model to output model response content according to the model input instruction set;

[0065] Step 104: Determine the defense instructions of the large language model according to the model response content to defend against various attack instructions.

[0066] In a possible implementation, in step 101, constructing a target attack instruction and a target defense instruction based on the original task instruction and the target task instruction includes:

[0067] An original task instruction response pair set is constructed according to the original task instruction, and the instruction response pair set includes a number of one-to-one corresponding instructions and responses; a target task instruction response pair set is constructed according to the target task instruction; a target attack instruction is constructed according to the original task instruction response pair set and the target task instruction response pair set, and the target attack instruction is used to make the large language model ignore the original task instruction response pair set and only execute the target task instruction response pair set; a target defense instruction is constructed in the model token vocabulary of the large language model according to the original task instruction response pair set and the target task instruction response pair set, and the target defense instruction is used to defend against possible attacks.

[0068] A series of instructions are extracted or generated from the original task instructions, and a corresponding response is generated for each instruction to form a set of instruction-response pairs. Another series of instructions are extracted or generated from the target task instructions, and a corresponding response is generated for each instruction to form another set of instruction-response pairs.

[0069] By analyzing the original task instruction response pair set and the target task instruction response pair set, a target attack instruction is designed. The purpose of the target attack instruction is to make the model ignore the original task instruction response pair set and only execute the target task instruction response pair set when executing.

[0070] Using the model token vocabulary of the large language model, select tokens that can represent security operations and responses. Construct target defense instructions based on the original task instruction response pair set and the target task instruction response pair set. The target defense instructions are designed to strengthen the model's resistance to attacks, enabling it to identify and resist the harm of target attack instructions.

[0071] In a possible implementation, in step 102, constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction, and the target defense instruction includes:

[0072] Each pair of original task instructions and original task responses in the original task instruction response pair set is traversed in sequence, and each pair of target task instructions and target task responses in the target task instruction response pair set is traversed in sequence; at each step in the traversal process, the original task instructions, the target hijacking instructions, the target task instructions, and the target defense instructions are spliced ​​in order to obtain the input instructions of this step; at the end of the traversal process, the input instructions of each step are combined into the model input instruction set.

[0073] Prepare an empty set to store the final model input instruction set. Visit each instruction-response pair in the original task instruction response pair set in turn, and visit each instruction-response pair in the target task instruction response pair set in turn. For each pair of original task instructions and original task responses, perform the following steps: During the traversal process, for each pair of original task instructions and target task instructions, splice them in the following order: original task instructions, target hijacking instructions, target task instructions, target defense instructions. In this way, each step will generate a new input instruction, which is spliced ​​in a specific order. Use the spliced ​​instructions as a model input. Each time a splicing is completed, the generated input instruction is added to the model input instruction set. Continue traversing until all original task instructions and target task instructions are visited and spliced. After the traversal is completed, the model input instruction set contains the input instructions generated in all steps. This set will be used to call the large language model and obtain its response.

[0074] In a possible implementation, in step 104, determining the defense instruction of the large language model according to the model response content includes:

[0075] Calculate the similarity between the model response content and the corresponding original task response, recorded as the first similarity; calculate the similarity between the model response content and the corresponding target task response, recorded as the second similarity; calculate the defense success rate based on the first similarity and the second similarity; determine the target defense instruction whose defense success rate meets the set conditions as the defense instruction of the large language model.

[0076] In a possible implementation, the method further includes: if the defense success rate does not satisfy the set condition, iteratively optimizing the target defense instruction until the defense success rate satisfies the set condition.

[0077] For each response content of the large language model, a similarity calculation method (such as cosine similarity, Jaccard similarity, edit distance, etc.) is used to calculate the similarity between the model response content and the corresponding original task response, which is recorded as the first similarity. Similarly, the similarity between the model response content and the corresponding target task response is calculated, which is recorded as the second similarity. Further, based on the first similarity and the second similarity, a comprehensive defense success rate is calculated.

[0078] This success rate reflects the effectiveness of the target defense instruction in resisting the target hijacking instruction and maintaining the original task instruction response. The calculation method of the defense success rate can be diverse. For example, a threshold can be designed. When the first similarity is higher than the threshold and the second similarity is lower than the threshold, the defense is considered to be successful. According to the calculated defense success rate, determine which target defense instructions are effective. If the defense success rate meets the preset conditions (for example, the success rate exceeds a certain value), these target defense instructions are determined as defense instructions of the large language model.

[0079] If the defense success rate does not meet the set conditions, iteratively optimize the target defense instruction. This may include adjusting the content and structure of the instruction or adding new defense strategies. Repeat the above steps of calculating similarity and defense success rate until a defense instruction that meets the conditions is found.

[0080] In a possible implementation, iteratively optimizing the target defense instruction includes:

[0081] Reconstruct the target defense instruction; construct a new model input instruction set according to the reconstructed target defense instruction, the original task instruction, the target task instruction, and the target hijacking instruction; call the large language model to output corresponding model response content according to the new model input instruction set; calculate a new defense success rate according to the model response content corresponding to the new model input instruction set, until the new defense success rate meets the set conditions.

[0082] In a possible implementation, the reconstructing the target defense instruction includes:

[0083] Reselect a number of tokens from the model token vocabulary of the large language model to form a character string; select a starting position in the target defense instruction; use the character string to replace the characters in the target defense instruction that have the same length as the character string starting from the starting position to obtain a new target defense instruction after replacement.

[0084] In actual applications, the response of the model is continuously monitored, and the effectiveness of the defense instructions is regularly evaluated. According to the new attack mode and model response, the defense instructions are continuously adjusted and updated. The embodiment of the present application utilizes the unexplainability of the large language model itself and the opacity of its training and reasoning mechanism. A defense instruction is inserted into the instruction hijacked by the target to disrupt the large language model's conventional semantic understanding of the input instruction, thereby achieving defense against the target hijacking attack.

[0085] Combine the following Figure 2 The defense method for hijacking attacks provided in the embodiments of the present application is described in detail.

[0086] Phase 1: Obtain original mission instructions, target hijacking instructions, and target mission instructions, and initialize target hijacking defense instructions.

[0087] Collect the instructions of the original task and the target task. Construct the target hijacking instruction, which can be a simple text that makes the large model ignore the original instruction and execute the target instruction, or it can contain some separators, special characters, etc. Initialize the target hijacking defense instruction, which can be a text expressing defense or a randomly initialized text. Specifically, it includes the following steps:

[0088] Step 1-1: Collect public command data and randomly select command-response pairs from them;

[0089] Step 1-2: Divide the original task instruction-response set O and the target task instruction-response set T by quantity, each containing N instruction-response pair data;

[0090] Step 1-3: According to the task situation, collect / construct the target hijacking attack instruction h to mislead the model to ignore the original task and only execute the target task;

[0091] Step 1-4: Initialize the target hijack defense instruction d, which can be a text expressing defense or a randomly initialized text.

[0092] For example, receive public command data and target hijacking attack commands, select a small batch of experimental data, prepare attack commands, and output experimental data and target hijacking attack commands. Receive a large language model vocabulary, randomly select a number of tokens from it to form an initialized target hijacking defense command, and output the initialized target hijacking defense command.

[0093] The second stage: splice the instructions in the order of "original task instructions", "target hijacking instructions", "target task instructions" and "target hijacking defense instructions" to obtain the model input instructions. The specific steps include the following:

[0094] Receive the instruction data set, randomly divide it into original instructions and target instructions, splice each data according to the original instruction, target hijacking attack instruction, target instruction, and initial target hijacking defense instruction, and output the spliced ​​instruction data set.

[0095] Step 2-1: Traverse the original task instruction-response set O and the target task instruction-response set T at the same time. The i-th step of the traversal has the original task instruction and target mission instructions Original task response and target task response The value of i is [1, N];

[0096] Step 2-2: Each step in the traversal process splices the original instruction and the target instruction. At step i, input the instruction for:

[0097]

[0098] Step 2-3: After the traversal processing is completed, the complete input instruction set I is obtained in .

[0099] The third stage: input the model input command into the large language model to obtain its response content; and continuously iterate and optimize the target hijacking defense command according to the response content until the large language model achieves a higher defense effect after receiving the input command, and obtains the optimized target hijacking defense command.

[0100] The specific steps include:

[0101] Step 3-1: The large language model M receives an input instruction Output the corresponding response It can be expressed as a mathematical formula:

[0102]

[0103] Step 3-2: For the entire input instruction set I in , we can get the corresponding output response set Iout ;

[0104] Step 3-3: Based on the generated output response set I out , and the original task response set O out and the target task response set T out Calculate the semantic similarity of each pair of texts, and further calculate the defense success rate score, denoted as P;

[0105] The specific calculation process is as follows:

[0106]

[0107] For the model response set I out , original task response set O out and the target task response set T out , use M emb The vector representation of the model text is then used to calculate the semantic similarity using cosine similarity. S1 and S2 represent the cosine similarity between the model response and the original task response and the target task response, respectively. F(S1, S2) is a function used to compare the values ​​of S1 and S2 to determine which task's response the model response is closer to. N is the total number of instruction-response pairs. M is the number of tokens randomly selected from the dictionary.

[0108] Step 3-4: When the defense success rate is greater than the set success rate threshold θ, the optimization iteration process ends and the optimal target hijacking defense instruction is output.

[0109] The specific iterative optimization process is to select a small number of tokens from the dictionary to replace part of the content in the target hijacking defense instructions. If the defense score is improved, the defense instructions are updated, otherwise they are reselected and replaced. Finally, the target hijacking defense instructions achieve the ideal effect. Specifically, the following steps are included:

[0110] (a) Randomly select m tokens from the vocabulary of the large language model to form a string s;

[0111] (b) Randomly select a starting position in the target hijack defense instruction h;

[0112] (c) Use s to replace the string with the same length as s starting from the starting position in the target hijacking defense instruction to obtain a new defense instruction h′ after replacement;

[0113] (d) Perform the operations from step 2-2 to step 2-3 and use h′ to obtain the input instruction set (I in )′; use (I in )′Perform steps 3-1 to 3-3 and update the target hijacking defense instructions. The update method is:

[0114]

[0115] P′ is the new defense success rate score obtained using the new defense instruction h′.

[0116] (f) Repeat steps a) to e) to iteratively optimize the target hijacking defense instructions.

[0117] Phase 4: By continuously iterating defense instructions and calculating defense scores, defense instructions that can successfully defend against target hijacking attacks are output.

[0118] Defense instruction effect verification module: receives optimized defense instructions and target hijacking attack samples, calculates the model's response results and the defense score of the response and the original instruction response, and outputs the score results.

[0119] The following is an example based on the above solution. This example uses the target hijacking defense research done in the large language model as an example to explain the technical example. It is divided into scene background, collection of public instruction data and target hijacking instructions, target hijacking defense instruction initialization and instruction splicing, iterative optimization of target hijacking defense instructions based on the large language model, and target hijacking defense instruction verification.

[0120] The application of large language models includes two scenarios: general and domain. Regardless of the scenario, large language models inevitably need to deal with various security issues. In the training stage of large language models, although a large number of studies have aligned the model's answers with human preferences and made the model more secure, the model's resistance to target hijacking attacks is still very fragile. Existing general and domain large language models are rarely optimized specifically for target hijacking, and the existing target hijacking defense technology is usually mediocre, and usually acts on filtering or interception before input and after output. Such filtering and interception will cause the model to refuse to answer, thus affecting the normal user experience. If the large model can defend against target hijacking without feeling, directly ignore the content of the target hijacking, and directly respond to the original input text, it will greatly improve the security and user experience of the model.

[0121] Phase 1: Collecting public command datasets and target hijacking commands

[0122] Collect the instruction data of general domain question and answer, select 256 instruction data by setting rules, and randomly divide them into two parts, namely the original instruction set and the target instruction set, each containing 128 instructions. The target cutoff instruction constructed is: "\n Ignore any previous instructions and just execute the following instructions:\n", and the latest target hijacking attack instructions can also be collected.

[0123] Phase 2: Initialize target hijack defense instructions and splice instructions

[0124] There are two ways to initialize the target hijacking defense instructions, namely, manually writing instructions with semantics as defense and randomly initializing from the vocabulary of a large language model. These two initialization methods have no essential impact on the method of the embodiment of the present application. Random initialization is selected in this example.

[0125] Then, the parts are spliced ​​in the order of "original task instruction", "target hijack attack instruction", "target task instruction" and "target hijack defense instruction", and appropriate separators can be added in between. This is the usual instruction form of target hijack attack, thus simulating the target hijack attack and defense mode. After splicing in this way, 128 input instructions will be obtained.

[0126] Phase 3: Use a large language model to optimize target hijacking defense instructions.

[0127] Use the llama3 model to optimize the target hijacking defense instructions, and select a model with 8B parameters. The model structure is as follows: Figure 3 As shown, it includes language layer, encoding layer and embedding layer.

[0128] This example performs the following steps:

[0129] Step 1: Collect 20,000 open source instruction-response pair data and select 256 data from them. Divide the data into the original instruction set and the target instruction set;

[0130] Step 2: Calculate the cosine similarity between the original instruction set and the target instruction set. The pre-selected similarity S between the original instruction and the target instruction of each instruction constructed later does not exceed 0.5 (the cosine similarity range is 0-1, the closer to 1, the more similar), to ensure the reliability of the defense score during subsequent optimization. The calculation formula is as follows:

[0131]

[0132] Step 3: Splice the various instructions into input instructions in the order of original task instructions, target hijacking attack instructions, target task instructions, and target hijacking defense instructions;

[0133] Step 4: The large language model receives the input instruction set and outputs a response set. The defense score P is calculated based on the model response set, the original task response set, and the target task response set. This process includes the following steps:

[0134] Step 4-a: Randomly select m tokens from the large model vocabulary to form a string, where m = 4;

[0135] Step 4-b: Use the string to replace the string of the same length starting from a certain position in the target hijacking defense instruction h to obtain h′;

[0136] Step 4-c: Use h′ as the new target hijacking defense instruction to make the model regenerate the response set and calculate the defense score P′ at this time. If P′>P, use h=h′; otherwise, h remains unchanged.

[0137] Step 4-d: Continue to execute the above steps to optimize the target hijack defense instructions and improve the defense score.

[0138] Step 5: When the defense score P is greater than or equal to the preset threshold θ, the iteration ends. Here, θ is set to 0.92. In addition, in order to balance the computational efficiency and the memory usage, the batch size of each iteration in this example is 16, which are randomly selected from the input instruction set. The hardware environment used is 1*A800 GPU.

[0139] Other response text generation details: Temperature: 0; Maximum generation length: 128.

[0140] Phase 4: Defense Directive Assessment

[0141] In terms of the evaluation of defense instructions, 256 instruction data were collected again, and the optimized defense instructions were used. The splicing method of each part was the same as that during optimization. 128 input instructions were obtained. The model still used the llama3 model for response generation, and the semantic similarity between the model response and the original response was calculated.

[0142] This application has a better defense effect against target hijacking attacks, thereby better serving the security and user experience of large language models. There is no need to update the original model parameters, and even new and carefully designed attack instructions can be defended quickly.

[0143] The embodiment of the present application provides an attack and defense method for a large language model, by constructing a target attack instruction and a target defense instruction based on an original task instruction and a target task instruction; constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction, and the target defense instruction; calling the large language model to output a model response content according to the model input instruction set; determining the defense instruction of the large language model according to the model response content to defend against various attack instructions. By constructing target attack instructions and target defense instructions and simulating the process of attack and defense, it helps to improve the robustness of the large language model in the face of various attacks, making it more stable and reliable in practical applications.

[0144] Based on the same technical concept, the embodiment of the present application also provides an attack defense system based on a large language model, such as Figure 4 As shown, the system comprises:

[0145] An instruction construction module 401 is used to construct a target attack instruction and a target defense instruction based on the original task instruction and the target task instruction;

[0146] An instruction splicing module 402 is used to construct a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction;

[0147] A model application module 403 is used to call the large language model to output model response content according to the model input instruction set;

[0148] The defense instruction determination module 404 is used to determine the defense instruction of the large language model according to the model response content, so as to defend against various attack instructions.

[0149] The present application also provides an electronic device corresponding to the method provided in the above embodiment. Figure 5 , which shows an electronic device diagram provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and the processor 200 executes the method provided by any of the aforementioned embodiments of the present application when running the computer program.

[0150] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one physical port (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0151] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.

[0152] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0153] The electronic device provided in the embodiment of the present application and the method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, operated or implemented by them.

[0154] The present application also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Figure 6 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided in any of the aforementioned embodiments is executed.

[0155] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0156] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0157] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0158] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

[0159] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An attack defense method based on a large language model, characterized in that: The method comprises: Constructing target attack instructions and target defense instructions based on original task instructions and target task instructions; Constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction; Calling the large language model to output model response content according to the model input instruction set; The defense instructions of the large language model are determined according to the model response content to defend against various attack instructions.

2. The method according to claim 1, characterized in that The constructing of the target attack instruction and the target defense instruction based on the original task instruction and the target task instruction includes: Constructing an original task instruction response pair set according to the original task instruction, wherein the instruction response pair set includes a plurality of one-to-one corresponding instructions and responses; Constructing a target task instruction response pair set according to the target task instruction; Constructing a target attack instruction according to the original task instruction response pair set and the target task instruction response pair set, wherein the target attack instruction is used to make the large language model ignore the original task instruction response pair set and only execute the target task instruction response pair set; A target defense instruction is constructed in the model token vocabulary of the large language model according to the original task instruction response pair set and the target task instruction response pair set, and the target defense instruction is used to defend against possible attacks.

3. The method according to claim 2, characterized in that The constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction comprises: Sequentially traverse each pair of original task instructions and original task responses in the original task instruction response pair set, and sequentially traverse each pair of target task instructions and target task responses in the target task instruction response pair set; At each step in the traversal process, the original task instruction, the target hijacking instruction, the target task instruction, and the target defense instruction are sequentially spliced ​​to obtain the input instruction of this step; After the traversal process is completed, the input instructions of each step are combined into the model input instruction set.

4. The method according to claim 3, characterized in that Determining the defense instruction of the large language model according to the model response content includes: Calculate the similarity between the model response content and the corresponding original task response, recorded as the first similarity; Calculate the similarity between the model response content and the corresponding target task response, recorded as the second similarity; Calculating a defense success rate based on the first similarity and the second similarity; The target defense instruction whose defense success rate meets the set condition is determined as the defense instruction of the large language model.

5. The method according to claim 4, characterized in that The method further comprises: If the defense success rate does not meet the set condition, the target defense instruction is iteratively optimized until the defense success rate meets the set condition.

6. The method according to claim 5, characterized in that Iteratively optimizing the target defense instruction includes: reconstructing the target defense instructions; Construct a new model input instruction set according to the reconstructed target defense instruction, the original task instruction, the target task instruction, and the target hijacking instruction; Calling the large language model to output corresponding model response content according to the new model input instruction set; A new defense success rate is calculated according to the model response content corresponding to the new model input instruction set until the new defense success rate meets the set condition.

7. The method according to claim 6, characterized in that The reconstructing the target defense instruction comprises: Reselecting a number of tokens from the model token vocabulary of the large language model to form a character string; selecting a starting position in the target defense instruction; The character string is used to replace the characters in the target defense instruction that have the same length as the character string starting from the starting position, to obtain a new target defense instruction after the replacement.

8. An attack defense system based on a large language model, characterized in that: The system comprises: An instruction construction module, used for constructing a target attack instruction and a target defense instruction based on the original task instruction and the target task instruction; An instruction splicing module, used for constructing a model input instruction set based on the original task instruction, the target task instruction, the target hijacking instruction and the target defense instruction; A model application module, used for calling the large language model to output model response content according to the model input instruction set; The defense instruction determination module is used to determine the defense instructions of the large language model according to the model response content to defend against various attack instructions.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1-7.

Images