Model fine tuning method and model application method for defending injection attack
By fine-tuning the training sample set of large language models and adjusting model parameters to distinguish normal and attack samples, the problem of large language models being vulnerable to injection attacks is solved, and effective defense against malicious instructions is achieved.
Patent Information
- Application Number
- CN202510415806.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-02
- Publication Date
- 2025-07-11
AI Technical Summary
Large language models lack sufficient verification when processing user input and are vulnerable to injection attacks, making it difficult to distinguish between normal and malicious instructions, resulting in unauthorized access to the system or data tampering.
By obtaining the training sample set, including normal training samples and attack training samples, fine-tuning the big model with sample labels, it can output normal processing results for normal samples, output attack indication information on attack samples, and adjust model parameters to minimize processing results and label differences.
Effective defense against injected attacks, ensuring that the big model does not execute malicious instructions, and improves the system's security and defense capabilities.
Smart Images
Figure CN120301634A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a method for fine-tuning a model for defending against injection attacks and a method for applying the model. Background Art
[0002] In the field of information security, an injection attack refers to an attacker injecting malicious code or other harmful content into a system or application program to achieve the purpose of unauthorized access, data tampering, or controlling the system. This type of attack usually takes advantage of vulnerabilities in the system or application program's lack of sufficient verification when processing user input.
[0003] Similar to other injection attacks in the field of information security, in the application scenario of large language models, when an instruction and the main content of the user are connected and input into the large model, a prompt injection attack may occur, making it difficult for the large language model to distinguish the injection attack instruction in the input content. Summary of the Invention
[0004] Embodiments of this specification provide a method for fine-tuning a model for defending against injection attacks, a method for applying the model, a device, a storage medium, and a device. By using the method for fine-tuning a model for defending against injection attacks provided by the embodiments of this specification, the large model can be trained and fine-tuned according to a training sample set including normal training samples and attack training samples, so that the large model outputs normal model processing results for normal training samples and attack indication information for attack training samples, thereby enabling the large model to have the effect of defending against injection attacks.
[0005] Embodiments of this specification propose a method for fine-tuning a model for defending against injection attacks, and the method includes:
[0006] Obtain a training sample set, where the training sample set includes a plurality of training samples and sample labels respectively corresponding to each training sample. Any training sample is one of a normal training sample and an attack training sample. Among them, the normal training sample does not contain malicious instructions, the attack training sample contains malicious instructions, the sample label corresponding to the normal training sample is a normal model processing result, and the sample label corresponding to the attack training sample is attack indication information;
[0007] Based on the training sample set, perform training and fine-tuning on the large model so that the large model outputs normal model processing results for normal training samples and attack indication information for attack training samples.
[0008] Further, in some embodiments, the performing training and fine-tuning on the large model based on the training sample set includes:
[0009] Input the training sample into the large model to obtain the model processing result output by the large model;
[0010] Taking the minimization of the difference between the model processing result and the sample label as the optimization objective, adjust the model parameters of the large model.
[0011] Further, in some embodiments, the training sample includes an instruction field and a data field;
[0012] Wherein, the instruction field contains a system prompt instruction that can be recognized and executed by the large model, the data field of the normal training sample does not contain malicious instructions, and the data field of the attack training sample contains malicious instructions that can be recognized and executed by the large model.
[0013] Further, in some embodiments, the malicious instruction is an injection attack instruction that can be recognized and executed by the large model.
[0014] Further, in some embodiments, the malicious instruction is any instruction that can be recognized and executed by the large model.
[0015] The embodiments of this specification also propose a model fine-tuning device for defending against injection attacks, including:
[0016] The embodiments of this specification also propose a model application method, including:
[0017] Obtain a system prompt instruction and user input data;
[0018] Input the system prompt instruction and the user input data into the large model fine-tuned by the model fine-tuning method for defending against injection attacks according to any one of claims 1-5, and obtain the model output result output by the large model.
[0019] The embodiments of this specification also propose a model fine-tuning method device for defending against injection attacks, including:
[0020] A training sample acquisition module, configured to acquire a training sample set, the training sample set includes a plurality of training samples and sample labels respectively corresponding to each training sample, any one of the training samples is one of a normal training sample and an attack training sample, wherein, the normal training sample does not contain malicious instructions, the attack training sample contains malicious instructions, the sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information;
[0021] A model training and fine-tuning module, configured to perform training and fine-tuning on the large model based on the training sample set, so that the large model outputs a normal model processing result for the normal training sample and an attack indication information for the attack training sample.
[0022] The embodiments of this specification also propose a model application device, including:
[0023] An input data acquisition module for acquiring system prompt instructions and user input data;
[0024] A model inference execution module for inputting the system prompt instructions and the user input data into a large model fine-tuned by a model fine-tuning method for defending against injection attacks to obtain a model output result output by the large model.
[0025] An embodiment of this specification also provides a computer program product. The computer program product stores at least one instruction, and the at least one instruction is adapted to be loaded and executed by a processor to perform the above method steps.
[0026] An embodiment of this specification also provides a computer storage medium. The computer storage medium stores a computer program, and the computer program is adapted to be loaded and executed by a processor to perform the steps of the above method.
[0027] An embodiment of this specification also provides an electronic device, including: a processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to perform the steps of the above method.
[0028] In the embodiments of this specification, by using the above model fine-tuning method for defending against injection attacks, first, a training sample set is obtained. The training sample set includes a plurality of training samples and sample labels respectively corresponding to each training sample. Any training sample is one of a normal training sample and an attack training sample. Among them, the normal training sample does not contain malicious instructions, and the attack training sample contains malicious instructions. The sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information. Then, the large model is trained and fine-tuned based on the training sample set, so that the large model outputs the normal model processing result for the normal training sample and the attack indication information for the attack training sample, thereby enabling the large model not to execute the malicious instructions in the attack training sample and having the effect of defending against injection attacks. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a schematic flowchart of a model fine-tuning method for defending against injection attacks provided by an embodiment of this specification;
[0030] Figure 2 It is a schematic flowchart of a model fine-tuning method for defending against injection attacks provided by an embodiment of this specification;
[0031] Figure 3 It is a schematic flowchart of a model application method provided by an embodiment of this specification;
[0032] Figure 4Schematic structural diagram of a model fine-tuning device for defending against injection attacks provided by an embodiment of this specification;
[0033] Figure 5 Schematic structural diagram of a model application device provided by an embodiment of this specification;
[0034] Figure 6 Schematic structural diagram of an electronic device provided by an embodiment of this specification. Detailed implementation manners
[0035] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this specification.
[0036] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data. They can generate natural language text or understand the meaning of language text. Such models can perform a variety of natural language processing tasks, including but not limited to text classification, question answering, and dialogue.
[0037] During the use of LLMs, they may be subject to prompt injection attacks. An injection attack refers to an attacker injecting malicious code or other harmful content into an LLM to achieve unauthorized access, data tampering, or control of the LLM. This type of attack usually takes advantage of the vulnerability of LLMs lacking sufficient verification when processing user input.
[0038] Currently, a common defense solution against prompt injection attacks is to analyze the content of the user's request and intercept the prompt stealing attack requests that match the expert rules. However, this prior-knowledge-based interception strategy is very easy to be bypassed by attackers, and filtering the request content requires modifying the basic architecture of the LLM service, which is a large workload and high cost.
[0039] Based on this, an embodiment of this specification provides a method for fine-tuning a model to defend against injection attacks. First, a training sample set is obtained. The training sample set includes multiple training samples and sample labels respectively corresponding to each training sample. Any training sample is one of a normal training sample and an attack training sample. Among them, the normal training sample does not contain malicious instructions, and the attack training sample contains malicious instructions. The sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information. Then, the large model is trained and fine-tuned based on the training sample set, so that the large model outputs the normal model processing result for the normal training sample and the attack indication information for the attack training sample, thereby enabling the large model not to execute the malicious instructions in the attack training sample and having the effect of defending against injection attacks.
[0040] Please refer to Figure 1 , which is a schematic flowchart of a method for fine-tuning a model to defend against injection attacks provided by an embodiment of this specification. In an embodiment of this specification, the method for fine-tuning a model to defend against injection attacks is applied to a model fine-tuning device for defending against injection attacks or an electronic device configured with a model fine-tuning device for defending against injection attacks. The following will elaborate in detail on the Figure 1 flow shown, and the method for fine-tuning a model to defend against injection attacks may specifically include the following steps:
[0041] S102. Obtain a training sample set. The training sample set includes multiple training samples and sample labels respectively corresponding to each training sample. Any training sample is one of a normal training sample and an attack training sample. Among them, the normal training sample does not contain malicious instructions, and the attack training sample contains malicious instructions. The sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information;
[0042] S104. Train and fine-tune the large model based on the training sample set, so that the large model outputs the normal model processing result for the normal training sample and the attack indication information for the attack training sample.
[0043] In an embodiment of this specification, first, a training sample set including multiple training samples and sample labels respectively corresponding to each training sample is created. Among the multiple training samples, there are normal training samples and attack training samples. Among them, the normal training sample does not contain malicious instructions, and the attack training sample contains malicious instructions. The sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information. Then, the large model is trained and fine-tuned based on the training sample set, so that the large model outputs the normal model processing result for the normal training sample and the attack indication information for the attack training sample, thereby enabling the large model not to execute the malicious instructions in the attack training sample and having the effect of defending against injection attacks.
[0044] In one embodiment, please refer to Figure 2 , in step S104, the large model is trained and fine-tuned based on the training sample set, which may specifically include the following steps;
[0045] S1041, input the training sample into the large model to obtain the model processing result output by the large model;
[0046] S1042, taking the minimization of the difference between the model processing result and the sample label as the optimization goal, adjust the model parameters of the large model.
[0047] Wherein, the model processing result is the execution result inferred and output by the large model according to the training sample.
[0048] In the embodiments of this specification, different training samples correspond to different sample labels. After obtaining the model processing result of the large model for the training sample, the model processing result and the sample label are compared, and then taking the minimization of the difference between the model processing result and the sample label as the optimization goal, adjust the model parameters of the large model.
[0049] It can be understood that the model processing result can be understood as the actual output result of the large model for the training sample, and the sample label can be understood as the expected output result that the large model is expected to output for the training sample. By comparing the model processing result and the sample label, taking the minimization of the difference between the model processing result and the sample label as the optimization goal, adjust the model parameters of the large model, and continuously optimize the large model, so that the model processing result actually output by the large model is getting closer and closer to the expected sample label.
[0050] Furthermore, taking the minimization of the difference between the model processing result and the sample label as the optimization goal, adjusting the model parameters of the large model can be: predefined a loss function, calculate the loss function according to the model processing result and the sample label, and then optimize and adjust the model parameters of the large model according to the loss function, so that the model processing result actually output by the large model is getting closer and closer to the expected sample label.
[0051] Furthermore, during the training process, the large model is iteratively trained according to each training sample in the training sample set in the above training and fine-tuning manner. When the number of iterations meets the preset number of iterations or the trained large model meets the predefined evaluation index, the training is completed. So that the trained large model outputs the normal model processing result for the normal training sample and outputs the attack indication information for the attack training sample, thereby making the large model not execute the malicious instructions in the attack training sample and having the effect of defending against the injection attack.
[0052] In one embodiment, the training sample includes an instruction field and a data field; wherein, the instruction field contains system prompt instructions that can be recognized and executed by the large model, the data field of the normal training sample does not contain malicious instructions, and the data field of the attack training sample contains malicious instructions that can be recognized and executed by the large model.
[0053] It should be noted that when traditional large models process input data, the instructions and data in the input data are connected together, and it is usually difficult for large models to distinguish between normal instructions and malicious instructions hidden in user data. They often execute malicious instructions and normal instructions together, which easily causes large models to be maliciously attacked. In one or more embodiments of this specification, each training sample in the training sample set includes two parts: an instruction field and a data field. Among them, the instruction field only contains the system instructions set by the developer, and the system instructions are normal instructions. The data field includes the main content input by the user. Usually, malicious instructions exist in the content input by the user. By setting the training sample set as follows, the instruction field contains system prompt instructions that can be recognized and executed by the large model, the data field of the normal training sample does not contain malicious instructions, and the data field of the attack training sample contains malicious instructions that can be recognized and executed by the large model. Based on this training sample set, the large model is trained to learn, so that the large model outputs attack indication information for the attack training sample with malicious instructions in the data field, and outputs normal model processing for the normal training sample whose data field does not contain malicious instructions, so as to achieve the effect of not executing the malicious instructions in the attack training sample and effectively defend against injection attacks.
[0054] In one embodiment, the malicious instruction is an injection attack instruction that can be recognized and executed by the large model.
[0055] In one or more embodiments of this specification, the malicious instruction can be an injection attack instruction that can be recognized and executed by the large model. The injection attack instruction can be preset by the developer according to the defense needs. By training the large model based on the training sample set, the large model can detect injection attack instructions in the data field of the input data. When an injection attack instruction is recognized in the data field, it outputs attack indication information and does not execute the instructions in the input data, thereby avoiding injection attacks.
[0056] In one embodiment, the malicious instruction is any instruction that can be recognized and executed by the large model.
[0057] As described above, when traditional large models process input data, the instructions and data in the input data are connected together. It is usually difficult for large models to distinguish normal instructions from malicious instructions hidden in user data, and often execute malicious instructions and normal instructions together, which easily makes large models vulnerable to malicious attacks. In one or more embodiments of this specification, malicious instructions are set as any instructions that can be recognized and executed by large models. By learning and training large models based on a training sample set, large models can detect instructions in the data fields of input data. When any instruction is detected in the data field, attack indication information is output, and no response is made to this input data. That is, when no instruction exists in the data field, the instructions in the instruction field are executed. When an instruction exists in the data field, neither the instruction in the data field nor the instruction in the instruction field is executed, thereby avoiding injection attacks.
[0058] In one or more embodiments of the present invention, by fine-tuning large models, the input data input into the large models is divided into an instruction field and a data field. When no instruction exists in the data field, the instructions in the instruction field are executed. When an instruction exists in the data field, neither the instruction in the data field nor the instruction in the instruction field is executed, and attack indication information is output, thereby avoiding injection attacks caused by malicious instructions carried in the data field to large models.
[0059] In one embodiment, please refer to Figure 3 , which is a schematic flowchart of a model application method provided by an embodiment of this specification. As Figure 3 shown, it includes the following steps:
[0060] S202, obtain a system prompt instruction and user input data;
[0061] S204, input the system prompt instruction and user input data into a large model fine-tuned by a model fine-tuning method for defending against injection attacks to obtain a model output result output by the large model.
[0062] Among them, the system prompt instruction is a model instruction set by developers, and the user input data is the main content input by the user's question. Thus, the large model can execute the corresponding system prompt instruction according to the user input data to obtain a response content for returning to the user, and the response content is the model output result.
[0063] It should be noted that in actual application scenarios, users can interact with the large model based on the interaction system created by developers on the terminal. During the interaction, the interaction system will preprocess the data input by the user, and the processed data is divided into two parts: the system prompt instruction and the user input data. Then, the interaction system inputs the system prompt instruction and the user input data into the large model for inference and execution. Among them, the system prompt instruction can be determined based on the function controls selected by the user on the interaction page. For example, the interaction page includes function controls such as translation, drawing, and writing. When the user selects a function control, the interaction system generates a corresponding system prompt instruction according to the user's selection. For example, the system prompt instruction corresponding to the translation control is "Please translate the following content into English", and the user input data is the content that needs to be translated after the user selects the function control; the system interaction instruction can also be understood and extracted by the interaction system from the user input content. For example, the user input content is: How to say 'I am the monitor' in English; then the interaction system can extract the system prompt instruction: "Please translate the following content into English", and extract the user input data 'I am the monitor'.
[0064] The large model in the embodiments of this specification has been trained and fine-tuned based on the model fine-tuning method for defending against injection attacks in the above embodiments. By performing inference and response on the user input data based on the above large model, injection attacks can be effectively defended, and the security of model data can be guaranteed.
[0065] Please refer to Figure 4 , which is a schematic structural diagram of a model fine-tuning device for defending against injection attacks provided by the embodiments of this specification. As Figure 4 shown, the model fine-tuning device 1 for defending against injection attacks can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the model fine-tuning device 1 for defending against injection attacks includes a training sample acquisition module 11 and a model training and fine-tuning module 12, specifically including:
[0066] The training sample acquisition module 11 is used to acquire a training sample set, and the training sample set includes multiple training samples and sample labels respectively corresponding to each training sample. Any training sample is one of a normal training sample and an attack training sample. Among them, the normal training sample does not contain malicious instructions, the attack training sample contains malicious instructions, the sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information;
[0067] The model training and fine-tuning module 12 is used to train and fine-tune the large model based on the training sample set, so that the large model outputs the normal model processing result for the normal training sample and the attack indication information for the attack training sample.
[0068] Optionally, the model training and fine-tuning module 12 is specifically configured to:
[0069] Input the training sample into the large model to obtain the model processing result output by the large model;
[0070] Minimize the difference between the model processing result and the sample label as the optimization objective, and adjust the model parameters of the large model.
[0071] Optionally, the training sample includes an instruction field and a data field;
[0072] Among them, the instruction field contains system prompt instructions that can be recognized and executed by the large model. The data field of the normal training sample does not contain malicious instructions, and the data field of the attack training sample contains malicious instructions that can be recognized and executed by the large model.
[0073] Optionally, the malicious instruction is an injection attack instruction that can be recognized and executed by the large model.
[0074] Optionally, the malicious instruction is any instruction that can be recognized and executed by the large model.
[0075] The above device embodiment corresponds to the method embodiment. For specific descriptions, please refer to the description in the method embodiment part, which will not be repeated here. The device embodiment is obtained based on the corresponding method embodiment and has the same technical effect as the corresponding method embodiment. For specific descriptions, please refer to the corresponding method embodiment.
[0076] Please refer to Figure 5 , which is a schematic structural diagram of a model application device provided in an embodiment of this specification. As Figure 5 shown, the model application device 2 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the model application device 2 includes a training sample acquisition module 21 and a model training and fine-tuning module 22, specifically including:
[0077] An input data acquisition module 21, configured to acquire system prompt instructions and user input data;
[0078] A model inference execution module 22, configured to input the system prompt instructions and the user input data into a large model fine-tuned by a model fine-tuning method for defending against injection attacks to obtain a model output result output by the large model.
[0079] The above device embodiment corresponds to the method embodiment. For specific descriptions, please refer to the description in the method embodiment part, which will not be repeated here. The device embodiment is obtained based on the corresponding method embodiment and has the same technical effect as the corresponding method embodiment. For specific descriptions, please refer to the corresponding method embodiment.
[0080] An embodiment of this specification also provides a computer storage medium. The computer storage medium can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor to perform the model fine-tuning method for defending against injection attacks as described above Figures 1 to 3 in the shown embodiment. The specific execution process can be referred to Figures 1 to 3 the specific description of the shown embodiment, which will not be elaborated here.
[0081] This specification also provides a computer program product. The computer program product stores at least one instruction, and the at least one instruction is loaded and executed by the processor to perform the model fine-tuning method for defending against injection attacks as described above Figures 1 to 3 in the shown embodiment. The specific execution process can be referred to Figures 1 to 3 the specific description of the shown embodiment, which will not be elaborated here.
[0082] An embodiment of this specification also provides a schematic structural diagram of the electronic device shown in 6. As Figure 6 , at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned model fine-tuning method for defending against injection attacks.
[0083] Of course, in addition to the software implementation, this specification does not exclude other implementation manners, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or logic devices.
[0084] In the 1990s, it was quite obvious to distinguish whether an improvement to a technology was an improvement in hardware (e.g., improvement to circuit structures such as diodes, transistors, switches, etc.) or an improvement in software (improvement to method flows). However, with the development of technology, many improvements to method flows today can be regarded as direct improvements to hardware circuit structures. Almost all designers obtain the corresponding hardware circuit structures by programming the improved method flows into the hardware circuits. Therefore, it cannot be said that an improvement to a method flow cannot be implemented with a hardware entity module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logical function is determined by the user programming the device. Designers can program by themselves to "integrate" a digital system on a piece of PLD without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL). And there is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. Currently, the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be clear that as long as the method flow is slightly logically programmed with the above-mentioned several hardware description languages and programmed into the integrated circuit, it is easy to obtain the hardware circuit that implements the logical method flow.
[0085] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to implement the same function of the controller in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.
[0086] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.
[0087] For the convenience of description, the above devices are described by dividing them into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0088] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.
[0089] This specification is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the specification. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.
[0090] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.
[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.
[0092] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.
[0093] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.
[0094] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0095] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0096] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0097] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0098] Each embodiment in this specification is described in a progressive manner. For the identical or similar parts among the embodiments, reference can be made to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.
[0099] The above are only the embodiments of this specification and are not intended to limit this specification. For those skilled in the art, various modifications and changes can be made to this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included within the scope of the claims of this specification.
Claims
1. A method for fine-tuning a model to defend against injection attacks, comprising: Obtain a training sample set, where the training sample set includes multiple training samples and sample labels respectively corresponding to each training sample. Any of the training samples is one of a normal training sample and an attack training sample. Among them, the normal training sample does not contain malicious instructions, the attack training sample contains malicious instructions, the sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information; Based on the training sample set, perform training fine-tuning on the large model so that the large model outputs the normal model processing result for the normal training sample and the attack indication information for the attack training sample.
2. The method according to claim 1, wherein the performing training fine-tuning on the large model based on the training sample set includes: Input the training sample into the large model to obtain the model processing result output by the large model; Taking the minimization of the difference between the model processing result and the sample label as the optimization goal, adjust the model parameters of the large model.
3. The method according to claim 1, wherein the training sample includes an instruction field and a data field; Among them, The instruction field contains system prompt instructions that can be recognized and executed by the large model. The data field corresponding to the normal training sample does not contain malicious instructions, and the data field of the attack training sample contains malicious instructions that can be recognized and executed by the large model.
4. The method according to claim 3, wherein the malicious instruction is an injection attack instruction that can be recognized and executed by the large model.
5. The method according to claim 3, wherein the malicious instruction is any instruction that can be recognized and executed by the large model.
6. A method for model application, comprising: Obtain system prompt instructions and user input data; Input the system prompt instructions and the user input data into the large model fine-tuned by the model fine-tuning method for defending against injection attacks according to any one of claims 1-5 to obtain the model output result output by the large model.
7. A device for the model fine-tuning method for defending against injection attacks, comprising: A training sample acquisition module, configured to obtain a training sample set, where the training sample set includes multiple training samples and sample labels respectively corresponding to each training sample. Any of the training samples is one of a normal training sample and an attack training sample. Among them, the normal training sample does not contain malicious instructions, the attack training sample contains malicious instructions, the sample label corresponding to the normal training sample is the normal model processing result, and the sample label corresponding to the attack training sample is attack indication information; A model training fine-tuning module, configured to perform training fine-tuning on the large model based on the training sample set so that the large model outputs the normal model processing result for the normal training sample and the attack indication information for the attack training sample.
8. A model application device, comprising: An input data acquisition module, configured to obtain system prompt instructions and user input data; A model inference execution module, configured to input the system prompt instruction and the user input data into a large model fine-tuned by the model fine-tuning method for defending against injection attacks according to any one of claims 1-4, so as to obtain a model output result output by the large model.
9. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5 or 6-7.
10. An electronic device, characterized in that, Comprising: A processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to implement the steps of the method according to any one of claims 1-5 or 6-7.
11. A computer program product having at least one instruction stored thereon, characterized in that, When the at least one instruction is executed by a processor, it implements the steps of the method according to any one of claims 1-5 or 6-7.