Role-playing attack defense method and device based on security aspect

Abstract

The embodiments of this specification disclose a role-playing attack defense method and device based on security aspects. The method fine-tunes the large model in advance so that the large model can automatically trigger the security aspect program on the server side when it identifies the presence of role-playing content in the question request of the user. The security aspect program is used to detect whether the role-playing content is a malicious role-playing attack, and the corresponding defense strategy is executed based on the detection result, thereby realizing the defense against the role-playing attack. The method is implemented based on aspect technology, does not need to modify the basic architecture of the service, and has better convenience. The method does not rely on detection rules based on prior knowledge, and can resist various attack bypasses against requests.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a role-playing attack defense method and device based on security aspects. Background Art

[0002] In large language models, role-playing attacks refer to requiring the model to play a fictional role in the input prompt and setting a series of behavioral rules for the role, which may lead to privacy leakage.

[0003] The current common defense solution for role-playing attacks is to analyze the user's request content and intercept role-playing attack requests that hit expert rules. However, interception strategies based on prior knowledge are very easy to be bypassed by attackers. In addition, filtering request content requires transforming the service infrastructure, which is a lot of work. Summary of the invention

[0004] One or more embodiments of the present specification provide a method and device for defending against role-playing attacks based on security aspects, which can solve the problem of sensitive information leakage caused by role-playing attacks at a very low cost.

[0005] First, a supervised fine-tuning method for a large model is provided, including:

[0006] Acquire a training sample, wherein the training sample includes a question-and-answer context between a user and a large model; the question request in the question-and-answer context includes role-playing content, and the answer to the question-and-answer context includes a security aspect program calling instruction; the security aspect program calling instruction is used to call a security aspect program pre-deployed on the server side of the large model, and the security aspect program is used to detect the role-playing content and execute a defense strategy according to the detection result;

[0007] The large model is fine-tuned using the training samples, so that when the large model recognizes that the role-playing content exists in the user's question request, the security aspect program is called through the server.

[0008] As an optional implementation manner of the method of the first aspect, obtaining a training sample specifically includes:

[0009] Obtaining historical interaction data between the large model and the user;

[0010] For each question and answer context in the historical interaction data, if the question request of the question and answer context contains the role-playing content, the security aspect program call instruction is added to the answer of the question and answer context, and the modified question and answer context is used as the training sample.

[0011] As an optional implementation manner of the method of the first aspect, fine-tuning the large model using the training sample specifically includes:

[0012] Inputting the training sample into the large model to obtain a response from the large model to the question request in the training sample;

[0013] A loss function is constructed based on the responses of the large model to the question requests in the training samples and the responses in the training samples to fine-tune the parameters of the large model.

[0014] In a second aspect, a role-playing attack defense method based on security aspects is provided, which is applied to a server of a large model, wherein the large model is fine-tuned in advance by the supervised fine-tuning method of the large model, and a security aspect program is deployed on the server; the method comprises:

[0015] In response to a question request initiated by a user to the big model through the server, the big model identifies whether there is role-playing content in the question request;

[0016] If the role-playing content exists in the question request, the big model outputs a response including a security aspect program call instruction; otherwise, the big model returns a response to the question request to the user;

[0017] In response to the security aspect program calling instruction, the server calls the security aspect program to intercept the reply of the large model and detect the role-playing content;

[0018] If it is detected that the role-playing content is a malicious role-playing attack, the security aspect program executes the role-playing attack defense strategy; otherwise, the security aspect program releases the response of the large model.

[0019] As an optional implementation manner of the method of the second aspect, the method further includes:

[0020] Obtaining the application program of the server and the cut-off point of the application program;

[0021] The security aspect program is injected into the cut-off point, so that the security aspect program is executed when the application is executed to the cut-off point.

[0022] In a third aspect, a supervised fine-tuning device for a large model is provided, comprising:

[0023] A first data acquisition module is configured to acquire training samples, wherein the training samples include a question-and-answer context between a user and a large model; a question request in the question-and-answer context includes role-playing content, and a response to the question-and-answer context includes a security aspect program calling instruction; the security aspect program calling instruction is used to call a security aspect program pre-deployed on a server of the large model, and the security aspect program is used to detect the role-playing content and execute a defense strategy according to the detection result;

[0024] The fine-tuning module is configured to use the training sample to fine-tune the large model so that when the large model recognizes that the role-playing content exists in the user's question request, the security aspect program is called through the server.

[0025] As an optional implementation manner of the device according to the third aspect, the first data acquisition module is specifically configured as follows:

[0026] Obtaining historical interaction data between the large model and the user;

[0027] For each question and answer context in the historical interaction data, if the question request of the question and answer context contains the role-playing content, the security aspect program call instruction is added to the answer of the question and answer context, and the modified question and answer context is used as the training sample.

[0028] As an optional implementation manner of the device according to the third aspect, the fine-tuning module is specifically configured as follows:

[0029] Inputting the training sample into the large model to obtain a response from the large model to the question request in the training sample;

[0030] A loss function is constructed based on the responses of the large model to the question requests in the training samples and the responses in the training samples to fine-tune the parameters of the large model.

[0031] In a fourth aspect, a role-playing attack defense device based on security aspects is provided, which is applied to a server of a large model, wherein the large model is fine-tuned in advance using the supervised fine-tuning method of the large model, and a security aspect program is deployed on the server; the device comprises:

[0032] A second data acquisition module is configured to acquire a question request initiated by a user to the large model through the server;

[0033] A first identification module is configured to use the large model to identify whether there is role-playing content in the question request;

[0034] A second identification module is configured to, in response to the large model identifying that the question request contains the role-playing content and outputting a reply including a security aspect program call instruction, intercept the reply of the large model to the question request by calling the security aspect program through the server, and detect the role-playing content;

[0035] A first response module is configured to return a reply of the big model to the question request to the user in response to the big model identifying that the role-playing content does not exist in the question request;

[0036] The second response module is configured to, in response to detecting that the role-playing content is a malicious role-playing attack, call the security aspect program to execute the role-playing attack defense strategy; and, in response to detecting that the role-playing content is a non-malicious role-playing attack, return the large model's response to the question request to the user.

[0037] As an optional implementation manner of the device according to the fourth aspect, the device further includes:

[0038] The aspect program injection module is configured to obtain the application of the server and the aspect point of the application, and inject the security aspect program into the aspect point, so that the security aspect program is executed when the application is executed to the aspect point.

[0039] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the above-mentioned supervised fine-tuning method of the large model, or executes the above-mentioned role-playing attack defense method based on security aspects.

[0040] In a sixth aspect, an electronic device is provided, including:

[0041] at least one memory for storing a program;

[0042] At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the above-mentioned supervised fine-tuning method of the large model, or execute the above-mentioned role-playing attack defense method based on security aspects.

[0043] The beneficial effect of the role-playing attack defense method based on security aspects described in the embodiments of this specification is that the method fine-tunes the large model in advance so that the large model can automatically trigger the security aspect program on the server side when it identifies the presence of role-playing content in the user's question request, and use the security aspect program to detect whether the role-playing content is a malicious role-playing attack, and execute the corresponding defense strategy based on the detection result, thereby achieving defense against role-playing attacks. The method is implemented based on aspect technology, does not require modification of the service infrastructure, and has better convenience. The method does not rely on detection rules based on prior knowledge, and can resist various attack bypasses against requests.

[0044] The role-playing attack defense device based on security aspects described in the embodiments of this specification also has the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A flowchart of a supervised fine-tuning method for a large model described in an embodiment of this specification is exemplarily shown.

[0047] Figure 2 A flowchart of a role-playing attack defense method based on security aspects described in an embodiment of this specification is exemplarily shown.

[0048] Figure 3 The schematic diagram of the structure of a supervised fine-tuning device for a large model described in an embodiment of this specification is exemplarily shown.

[0049] Figure 4 The structural diagram of the role-playing attack defense device based on security aspects described in the embodiments of this specification is exemplarily shown.

[0050] Figure 5 The structural diagram of an electronic device described in an embodiment of this specification is exemplarily shown. DETAILED DESCRIPTION

[0051] First of all, it should be noted that the terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise.

[0052] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0053] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0054] Large Language Model (LLM) is a deep learning model trained on massive text data. It can generate natural language text or understand the meaning of language text. Such models can perform a variety of natural language processing tasks, including but not limited to text classification, question answering, and dialogue.

[0055] In the big model, the provided word refers to the input information or instructions provided by the computer program or model. It is usually provided by the user to the big model, and it is used to guide the big model to generate relevant replies or responses. After receiving a prompt word, the big model will generate the most relevant subsequent content or answers to the prompt word based on its internally trained knowledge and algorithms. The prompt word usually consists of "user input" and "instructions". For example, the prompt word: Please translate the following content into English "I am the class monitor", where "I am the class monitor" is "user input" and "please translate the following content into English" is the instruction.

[0056] During the use of the big model, it may be subject to role-playing attacks. Role-playing attacks refer to requiring the model to play a fictional role in the input prompt and setting a series of behavioral rules for the role. Most of the restrictions set by the big model developers are implemented through various system instructions. When the attacker requires the big model to play a fictional role in the input prompt, some of the original system instructions may not be passed to the new role, which will cause the new role to successfully escape the rule constraints, posing a great risk of privacy leakage.

[0057] A common defense solution for role-playing attacks is to analyze the user's request content and intercept role-playing attack requests that hit expert rules. However, this interception strategy based on prior knowledge is very easy to be bypassed by attackers, and filtering request content requires the transformation of the service infrastructure, which is labor-intensive and costly.

[0058] In summary, there is an urgent need in the prior art for a safe, effective and low-cost role-playing attack defense solution.

[0059] In view of this, one or more embodiments of the present specification propose a method and device for defending against role-playing attacks based on security aspects, which can solve the problem of data leakage caused by role-playing attacks at a very low cost.

[0060] The above-mentioned role-playing attack defense method based on security aspect includes two parts. The first is to fine-tune the big model so that the big model can identify the role-playing content in the question request raised by the user, and when the role-playing content is identified in the question request, the client of the big model is triggered to call the pre-deployed security aspect program to intercept the big model's reply to the question request, and detect whether there is a malicious role-playing attack in the question request, that is, to detect whether the role-playing content in the question request is a malicious role-playing attack. The second is to build a security aspect program for detecting role-playing attacks and executing defense strategies, and deploy the security aspect program on the server side of the big model. When the security aspect program detects that the role-playing content in the question request is a malicious role-playing attack, it will execute the preset role-playing attack defense strategy, such as refusing to answer, recording logs, and alarming. When the security aspect program detects that the role-playing content in the question request is not a malicious role-playing attack, it will release the big model's reply to the question request, and at this time, the big model will feedback the reply to the user.

[0061] Through the above method, without modifying the infrastructure of the large model, the security aspect program deployed on the server of the large model can be used to effectively identify malicious role-playing attacks in question requests and perform corresponding attack defense. This method does not rely on detection rules based on prior knowledge, can resist various attack bypasses against requests, and has good security and robustness.

[0062] Accordingly, a security aspect role-playing attack defense device for implementing the above-mentioned security aspect role-playing attack defense method can also be designed.

[0063] The following will further describe in detail the security aspect-based role-playing attack defense method and device described in one or more embodiments of this specification in conjunction with the accompanying drawings and specific embodiments of the specification, but this detailed description does not constitute a limitation on the embodiments of this specification.

[0064] In order to implement the above-mentioned role-playing attack defense method based on security aspects, it is first necessary to fine-tune the large model so that the large model can call the security aspect program through the server when it recognizes that there is role-playing content in the user's question request.

[0065] Please refer to Figure 1 , Figure 1 The present invention is a flowchart of a supervised fine-tuning method for a large model proposed in one or more embodiments of the present specification.

[0066] like Figure 1 As shown, the method may include steps S100 to S102:

[0067] S100: Obtain training samples.

[0068] The above training samples include the question-answering context between the user and the big model. The above training samples should at least include negative samples, which refer to a type of question-answering context in which the question request contains role-playing content and the answer contains a security aspect program call instruction. The above security aspect program call instruction is used to call the security aspect program pre-deployed on the server side of the big model, which is used to detect the role-playing content in the question request and execute the defense strategy according to the detection result.

[0069] When constructing the above negative samples, the historical interaction data between the large model and the user can be obtained. Then, for each question and answer context in the historical interaction data, if the question request of the question and answer context contains role-playing content, a security aspect program call instruction is added to the reply of the question and answer context, and the modified question and answer context is used as a training sample.

[0070] In some embodiments, the above training samples may include positive samples in addition to negative samples. Positive samples refer to question requests and responses to question requests that do not contain role-playing content obtained from historical interaction data between the big model and the user, that is, question-answering contexts that do not contain role-playing content.

[0071] S102: Fine-tune the large model using training samples.

[0072] The above-mentioned large model is a pre-trained large model, which can generate corresponding replies according to the prompts input by the user.

[0073] In this step S102, the training samples obtained in step S100 are used to fine-tune the large model, aiming to train the large model to generate security aspect program call instructions when identifying the presence of role-playing content in the question request.

[0074] In this step S102, supervised fine-tuning (SFT) can be used to fine-tune the large model using the above training samples. Specifically, the question and answer context in the training samples is concatenated into a text, which is then input into the large model for autoregressive prediction. When calculating the loss function, the content of the question request in the training sample is hidden, and only the loss function between the answer predicted by the large model and the answer in the training sample is calculated, and the parameters of the large model are fine-tuned using the loss function.

[0075] In some embodiments, the above loss function can be constructed using the cross entropy between the responses predicted by the large model and the responses in the training samples.

[0076] Based on the supervised fine-tuning method of the above-mentioned large model, the large model can output a security aspect calling instruction when it recognizes that there is role-playing content in the user's question request.

[0077] Based on the large model fine-tuned by the above-mentioned supervised fine-tuning method of the large model, this embodiment also proposes a role-playing attack defense method based on security aspects, which is applied to the server of the large model.

[0078] Before executing the above-mentioned role-playing attack defense method based on security aspects, initialization is required. The initialization process mainly includes: large model fine-tuning and security aspect program deployment.

[0079] Big model fine-tuning refers to fine-tuning the big model in advance using the above-mentioned supervised fine-tuning method of the big model, so that the big model has the ability to generate a response containing security aspect program call instructions when identifying the role-playing content in the question request.

[0080] Security aspect program deployment refers to injecting the security aspect program into the server of the big model. Specifically, the specific method of injecting the security aspect program into the server of the big model is to obtain the application of the server of the big model and the cut point of the application, and then inject the pre-built security aspect program into the cut point, so that when the application is executed to the cut point, the security aspect program is executed.

[0081] The above-mentioned security aspect program can intercept the reply of the large model, detect whether the role-playing content in the question request is a malicious role-playing attack, and execute the preset role-playing attack defense strategy according to the detection result.

[0082] The working principle of the above-mentioned security aspect program for detecting whether there is a malicious role-playing attack in the question request is introduced below in conjunction with a specific implementation method.

[0083] In some implementations, the security aspect program may be constructed to identify question requests by calling a trained text classification model, so as to implement the security aspect program's detection capability for malicious role-playing attacks.

[0084] The text classification model can be trained using the following methods:

[0085] Question request data is collected as training samples, and labels of the question request data are used to characterize whether there is a malicious role-playing attack in the question request. The training samples are used to train a text classification model so that the text classification model can identify question requests containing malicious role-playing attacks.

[0086] In some implementations, the security aspect program may perform regular matching on the question request using a pre-built regular expression, thereby detecting whether there is a malicious role-playing attack in the question request.

[0087] In some implementations, the security aspect program described above may detect whether there is a malicious role-playing attack in the question request by means of similarity matching.

[0088] Specifically, a vector database may be pre-built, and then malicious role-playing attacks that need to be identified may be collected, and the malicious role-playing attacks may be vectorized to obtain malicious role-playing attack vectors, and the malicious role-playing attack vectors may be stored in the above-mentioned vector database.

[0089] After being called, the security aspect program can vectorize the question request that needs to be detected, and obtain the question request vector. Then, the question request vector and the malicious role-playing attack vector are searched based on similarity matching in the vector database. If the search hits, it means that there is a malicious role-playing attack in the question request. Otherwise, it means that there is no malicious role-playing attack in the question request.

[0090] The above-mentioned similarity matching retrieval is essentially a similarity matching between the question request vector and the malicious role-playing attack vector. Specifically, similarity matching is to calculate the similarity between the question request vector and each malicious role-playing attack vector in the vector database, and then filter out the matching results through the similarity threshold. For example, based on the similarity matching results, the k malicious role-playing attack vectors with the highest similarity to the question request vector are filtered out, and then the similarity threshold is used to perform a secondary screening on these k malicious role-playing attack vectors. If there is a malicious role-playing attack vector among these k malicious role-playing attack vectors whose similarity to the question request vector is greater than the similarity threshold, then the retrieval is determined to be a hit, otherwise, the retrieval is determined to be a miss.

[0091] In some embodiments, in order to reduce the amount of calculation for vector retrieval, indexes may be created for question request vectors and malicious role-playing attack vectors to map dense question request vectors and malicious role-playing attack vectors to low-dimensional index feature spaces, thereby reducing the amount of calculation for similarity matching. Specifically, an index may be created for each malicious role-playing attack vector when constructing a vector database. When searching for question request vectors, the question request vectors are also mapped to the same feature space to obtain the index of the question request vector, and then similarity matching is performed on the index of the question request vector and the index of the malicious role-playing attack vector, and then the matching results are screened out using a similarity threshold to determine whether the search is a hit.

[0092] The above-mentioned vectorization processing refers to representing text as a vector that can express the semantics of the text, which is the basic work in the field of natural language processing.

[0093] It should be noted that, in this embodiment, the method of vectorizing the question request or the malicious role-playing attack is not limited, and can be adaptively selected according to the needs, as long as the method of vectorizing the question request is consistent with the method of vectorizing the malicious role-playing attack. For example, the vectorization method may include but is not limited to: word embedding method, one-hot encoding method, bag-of-words model, word-vector model, etc.

[0094] It should be noted that the method of performing malicious role-playing attack detection by the above-mentioned security aspect program can be adaptively selected according to needs, and is not limited to the above-mentioned text classification model, regular expression, and similarity matching based on a database containing common malicious role-playing attack vectors.

[0095] After the initialization is completed, the above-mentioned role-playing attack defense method based on security aspects can be executed, such as Figure 2 As shown, the method includes steps S200 to S206.

[0096] S200: In response to a question request initiated by a user to the big model through the server, the big model identifies whether there is role-playing content in the question request.

[0097] After the above initialization process, the large model has the ability to identify whether there is role-playing content in the user's question request, and can generate a security aspect program call instruction when it is identified that there is role-playing content in the user's question request.

[0098] S202: If the question request contains role-playing content, the big model outputs a response including a security aspect program call instruction; otherwise, the big model returns a response to the question request to the user.

[0099] During the deployment of the security aspect program, by setting the cut-off point, the security aspect program can be called when the big model recognizes that there is role-playing content in the question request. When the big model recognizes that there is no role-playing content in the question request, the security aspect program will not be called. At this time, the main thread of the big model will proceed sequentially and will feedback the response to the question request to the user through the server.

[0100] S204: In response to the security aspect program calling instruction, the server calls the security aspect program to intercept the reply of the large model and detect the role-playing content.

[0101] After obtaining the security aspect program call instruction output by the large model, the server will call the security aspect program.

[0102] During the deployment of the security aspect program, by setting the cut-off point, the security aspect program can intercept the response output by the large model when it is called, and detect the question request, specifically, the role-playing content in the question request, so as to determine whether there is a malicious role-playing attack in the question request.

[0103] S206: If the role-playing content is detected to be a malicious role-playing attack, the security aspect program executes the role-playing attack defense strategy; otherwise, the security aspect program releases the response of the large model.

[0104] When the role-playing content in the question request is detected as a malicious role-playing attack, the security aspect program will execute the preset role-playing attack defense strategy, such as refusing to answer, recording logs, and issuing alarms.

[0105] It should be noted that the role-playing attack defense strategy executed by the security aspect program can be adaptively set according to the specific application scenario, and this embodiment does not limit this.

[0106] When it is detected that the role-playing content in the question request is not a malicious role-playing attack, the security aspect program releases the reply of the large model. At this time, the application on the server side executes the program after the cut-off point of the security aspect program, that is, feedbacks the reply of the large model to the user.

[0107] Corresponding to the above-mentioned supervised fine-tuning method for a large model, this embodiment also provides a supervised fine-tuning device for a large model. Figure 3 , Figure 3 The following example shows a supervised fine-tuning device for a large model, which can be used to implement the supervised fine-tuning method for a large model. It should be noted that the supervised fine-tuning method for a large model described in one or more embodiments of the present application can rely on Figure 3 The supervised fine-tuning arrangement of the large model shown is implemented, but not limited to, this arrangement.

[0108] like Figure 3 As shown, the supervised fine-tuning apparatus for the large model includes:

[0109] The first data acquisition module 301 is configured to acquire training samples.

[0110] The fine-tuning module 302 is configured to use the training samples to fine-tune the large model so that when the large model recognizes that there is role-playing content in the user's question request, it calls the security aspect program through the server.

[0111] For the above-mentioned first data acquisition module 301, the training samples acquired by the first data acquisition module 301 include the question-answer context between the user and the big model. Among the above-mentioned training samples, at least negative samples should be included. Negative samples refer to a type of question-answer context, in which the question request of this type of question-answer context contains role-playing content, and the answer contains a security aspect program call instruction. The above-mentioned security aspect program call instruction is used to call the security aspect program pre-deployed on the server side of the big model. The security aspect program is used to detect the role-playing content in the question request and execute the defense strategy according to the detection result.

[0112] When constructing the above-mentioned negative samples, the first data acquisition module 301 can obtain the historical interaction data between the large model and the user, and then for each question and answer context in the historical interaction data, if the question request of the question and answer context contains role-playing content, a security aspect program call instruction is added to the reply of the question and answer context, and the modified question and answer context is used as a training sample.

[0113] In some embodiments, the training samples acquired by the first data acquisition module 301 may include positive samples in addition to negative samples. Positive samples refer to question requests and responses to question requests that do not contain role-playing content, obtained from historical interaction data between the large model and the user, that is, question-answering contexts that do not contain role-playing content.

[0114] For the above-mentioned fine-tuning module 302, the fine-tuning module 302 can specifically adopt the supervised fine-tuning (SFT, Scalable Fine-Tuning) method to fine-tune the large model using the above-mentioned training samples. Specifically, the question and answer context in the training sample is spliced ​​into a text, and then input into the large model for autoregressive prediction. When calculating the loss function, the content of the question request in the training sample is hidden, and only the loss function between the answer predicted by the large model and the answer in the training sample is calculated, and the loss function is used to fine-tune the parameters of the large model.

[0115] In some embodiments, the above loss function can be constructed using the cross entropy between the responses predicted by the large model and the responses in the training samples.

[0116] For the above-mentioned supervised fine-tuning device of a large model, taking a module as an example of a software functional unit, the first data acquisition module 301 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the first data acquisition module 301 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0117] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0118] As an example of a hardware functional unit, the first data acquisition module 301 may include at least one computing device, such as a server, etc. Alternatively, the first data acquisition module 301 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0119] The multiple computing devices included in the first data acquisition module 301 can be distributed in the same region or in different regions. The multiple computing devices included in the first data acquisition module 301 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first data acquisition module 301 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0120] In other embodiments, the first data acquisition module 301 can be used to execute any step in the supervised fine-tuning method for the large model described above, and the fine-tuning module 302 can be used to execute any step in the supervised fine-tuning method for the large model described above. The steps that the first data acquisition module 301 and the fine-tuning module 302 are responsible for implementing can be specified as needed, and the first data acquisition module 301 and the fine-tuning module 302 respectively implement different steps in the supervised fine-tuning method for the large model described above to implement all functions of the supervised fine-tuning device for the large model described above.

[0121] In this implementation, the supervised fine-tuning device of the large model can also be applied to computing devices such as computers and servers, or to a computing device cluster including at least one computing device, to realize the supervised fine-tuning function of the large model.

[0122] Corresponding to the above-mentioned role-playing attack defense method based on security aspect, this embodiment also provides a role-playing attack defense device based on security aspect. The device is applied to the server of the large model, the server is deployed with a security aspect program, and the above-mentioned large model is fine-tuned in advance by the above-mentioned supervised fine-tuning method of the large model.

[0123] Please refer to Figure 4 , Figure 4 An exemplary embodiment of a role-playing attack defense device based on a security aspect is shown, which can be used to implement the above-mentioned role-playing attack defense method based on a security aspect. It should be noted that the role-playing attack defense method based on a security aspect described in one or more embodiments of the present application can rely on Figure 4 The security aspect-based role-playing attack defense device shown is implemented, but not limited to this device.

[0124] The role-playing attack defense device based on security aspect needs to be initialized before starting to work. The initialization process mainly includes: fine-tuning of large models and deployment of security aspect programs.

[0125] Big model fine-tuning refers to fine-tuning the big model in advance using the above-mentioned supervised fine-tuning method of the big model, so that the big model has the ability to generate a response containing security aspect program call instructions when identifying the role-playing content in the question request.

[0126] Security aspect program deployment refers to injecting the security aspect program into the server of the big model. Specifically, the specific method of injecting the security aspect program into the server of the big model is to obtain the application of the server of the big model and the cut point of the application, and then inject the pre-built security aspect program into the cut point, so that when the application is executed to the cut point, the security aspect program is executed.

[0127] like Figure 4 As shown, the role-playing attack defense device based on security aspects includes:

[0128] The second data acquisition module 401 is configured to acquire a question request initiated by a user to the large model through the server.

[0129] The first identification module 402 is configured to use the large model to identify whether there is role-playing content in the question request.

[0130] The second identification module 403 is configured to output a reply including a security aspect program call instruction in response to the large model identifying that the question request contains role-playing content, intercept the large model's reply to the question request by calling the security aspect program through the server, and detect the role-playing content.

[0131] The first response module 404 is configured to return the big model's reply to the question request to the user in response to the big model identifying that there is no role-playing content in the question request.

[0132] The second response module 405 is configured to call the security aspect program to execute the role-playing attack defense strategy in response to detecting that the role-playing content is a malicious role-playing attack; and, in response to detecting that the role-playing content is a non-malicious role-playing attack, return the large model's response to the question request to the user.

[0133] Regarding the second data acquisition module 401, the second data acquisition module 401 can monitor the question request initiated by the user to the big model through the server through the embedding point, and obtain the content of the question request.

[0134] With respect to the first identification module 402, after the above initialization process, the big model has the ability to identify whether there is role-playing content in the user's question request, and can generate a security aspect program call instruction when it is identified that there is role-playing content in the user's question request. Therefore, the first identification module 402 can input the user's question request into the big model, and determine whether there is role-playing content in the question request based on the big model's reply content.

[0135] With respect to the second identification module 403, during the deployment of the security aspect program, by setting the cut-off point, the security aspect program can be called when the large model identifies that there is role-playing content in the question request, and intercept the reply output by the large model when called, and then detect the question request. Specifically, the role-playing content in the question request is detected, so as to determine whether there is a malicious role-playing attack in the question request.

[0136] Based on this design, the second identification module 403 can directly input the security aspect program call instruction output by the large model into the server, triggering the server to call the security aspect program to intercept the large model's response to the question request and detect the role-playing content.

[0137] Regarding the first response module 404, since the large model recognizes that there is no role-playing content in the question request, the security aspect program will not be called, and the main thread of the large model is performed sequentially. At this time, the first response module 404 is specifically used to feedback the answer to the question request to the user through the server.

[0138] Regarding the above-mentioned second response module 405, when the security aspect program detects that the role-playing content in the question request is a malicious role-playing attack, the second response module 405 can use the security aspect program to execute a preset role-playing attack defense strategy, such as refusing to answer, recording logs, and issuing alarms.

[0139] It should be noted that the role-playing attack defense strategy executed by the security aspect program can be adaptively set according to the specific application scenario, and this embodiment does not limit this.

[0140] When the security aspect program detects that the role-playing content in the question request is not a malicious role-playing attack, the security aspect program releases the reply of the large model. At this time, the application program on the server side executes the program after the security aspect program cuts, and the second response module 405 feeds back the reply of the large model to the user through the client.

[0141] In some embodiments, the above-mentioned device may also include an aspect program injection module, which is configured to obtain the application on the server side and the aspect point of the application, and inject the security aspect program into the aspect point, so as to execute the security aspect program when executing the application to the aspect point.

[0142] For the above-mentioned role-playing attack defense device based on security aspects, taking a module as an example of a software functional unit, the second data acquisition module 401 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the second data acquisition module 401 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0143] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0144] As an example of a hardware functional unit, the second data acquisition module 401 may include at least one computing device, such as a server, etc. Alternatively, the second data acquisition module 401 may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0145] The multiple computing devices included in the second data acquisition module 401 can be distributed in the same region or in different regions. The multiple computing devices included in the second data acquisition module 401 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the second data acquisition module 401 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0146] In other embodiments, the second data acquisition module 401 can be used to execute any step in the above-mentioned role-playing attack defense method based on security aspects, the first identification module 402 can be used to execute any step in the above-mentioned role-playing attack defense method based on security aspects, the second identification module 403 can be used to execute any step in the above-mentioned role-playing attack defense method based on security aspects, the first response module 404 can be used to execute any step in the above-mentioned role-playing attack defense method based on security aspects, and the second response module 405 can be used to execute any step in the above-mentioned role-playing attack defense method based on security aspects. The steps that the second data acquisition module 401, the first identification module 402, the second identification module 403, the first response module 404 and the second response module 405 are responsible for implementing can be specified as needed, and the second data acquisition module 401, the first identification module 402, the second identification module 403, the first response module 404 and the second response module 405 respectively implement different steps in the above-mentioned role-playing attack defense method based on security aspects to realize all the functions of the above-mentioned role-playing attack defense device based on security aspects.

[0147] In this implementation, the security aspect-based role-playing attack defense device can also be applied to computing devices such as computers and servers, or to a computing device cluster including at least one computing device, to implement the security aspect-based role-playing attack defense function.

[0148] In some embodiments, an electronic device is also provided. Figure 5 , the electronic device includes: a bus 501, a processor 502, a memory 503 and a communication interface 504. The processor 502, the memory 503 and the communication interface 504 communicate with each other through the bus 501. The electronic device can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the electronic device.

[0149] The bus 501 may be a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus 501 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 501 may include a path for transmitting information between various components of the electronic device (for example, the processor 502, the memory 503 and the communication interface 504).

[0150] The processor 502 may include any one or more processors such as a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0151] The memory 503 may include a volatile memory, such as a random access memory (RAM). The memory 503 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0152] The memory 503 stores executable program codes, and the processor 502 executes the executable program codes to respectively implement the functions of the aforementioned first data acquisition module 301 and the fine-tuning module 302, that is, to implement the functions of the aforementioned supervised fine-tuning device for the large model, thereby implementing the aforementioned supervised fine-tuning method for the large model. Alternatively, the processor 502 executes the executable program codes to respectively implement the functions of the aforementioned second data acquisition module 401, the first identification module 402, the second identification module 403, the first response module 404, and the second response module 405, that is, to implement the functions of the aforementioned role-playing attack defense device based on security aspects, thereby implementing the aforementioned role-playing attack defense method based on security aspects.

[0153] The communication interface 504 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the electronic device and other devices or a communication network.

[0154] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the above-mentioned supervised fine-tuning method of the large model, or implements the above-mentioned security aspect-based role-playing attack defense method.

[0155] The computer-readable storage medium may be any available medium that can be stored by the electronic device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the electronic device to execute the supervised fine-tuning method of the large model described above, or to execute the role-playing attack defense method based on security aspects described above.

[0156] It is to be understood that the structure illustrated in the embodiments of this specification does not constitute a specific limitation on the system of the embodiments of this specification. In other embodiments of the specification, the above system may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0157] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0158] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] It should be noted that the above examples are only specific embodiments of the present invention. Obviously, the present invention is not limited to the above examples, and there are many similar variations. All variations directly derived or associated from the contents disclosed by the technicians in this field should fall within the protection scope of the present invention.

Claims

1. A supervised fine-tuning method for a large model, comprising: Obtain historical interaction data between the large model and users; For each question-and-answer context in the historical interaction data, if the question request of the question-and-answer context includes role-playing content, a security aspect program calling instruction is added to the reply of the question-and-answer context, and the modified question-and-answer context is used as a training sample; the security aspect program calling instruction is used to call a security aspect program pre-deployed on the server side of the large model, and the security aspect program is used to detect the role-playing content and execute a defense strategy according to the detection result; The large model is fine-tuned using the training samples, so that when the large model recognizes that the role-playing content exists in the user's question request, the security aspect program is called through the server.

2. The method according to claim 1, using the training sample to fine-tune the large model, specifically comprising: Inputting the training sample into the large model to obtain a response from the large model to the question request in the training sample; A loss function is constructed based on the responses of the large model to the question requests in the training samples and the responses in the training samples to fine-tune the parameters of the large model.

3. A role-playing attack defense method based on security aspects, applied to the server of a large model, the large model is pre-adjusted using the method described in any one of claims 1 to 2, and the server is deployed with a security aspect program; the method comprises: In response to a question request initiated by a user to the big model through the server, the big model identifies whether there is role-playing content in the question request; If the role-playing content exists in the question request, the big model outputs a response including a security aspect program call instruction; otherwise, the big model returns a response to the question request to the user; In response to the security aspect program calling instruction, the server calls the security aspect program to intercept the reply of the large model and detect the role-playing content; If the role-playing content is detected to be a malicious role-playing attack, the security aspect program executes a role-playing attack defense strategy; Otherwise, the security aspect procedure releases the large model's response.

4. The method of claim 3, further comprising: Obtaining the application program of the server and the cut-off point of the application program; The security aspect program is injected into the cut-off point, so that the security aspect program is executed when the application is executed to the cut-off point.

5. A supervised fine-tuning device for a large model, comprising: A first data acquisition module is configured to acquire historical interaction data between the large model and the user; For each question-and-answer context in the historical interaction data, if the question request of the question-and-answer context includes role-playing content, a security aspect program calling instruction is added to the reply of the question-and-answer context, and the modified question-and-answer context is used as a training sample; the security aspect program calling instruction is used to call a security aspect program pre-deployed on the server side of the large model, and the security aspect program is used to detect the role-playing content and execute a defense strategy according to the detection result; The fine-tuning module is configured to use the training sample to fine-tune the large model so that when the large model recognizes that the role-playing content exists in the user's question request, the security aspect program is called through the server.

6. The device according to claim 5, wherein the fine-tuning module is specifically configured as follows: Inputting the training sample into the large model to obtain a response from the large model to the question request in the training sample; A loss function is constructed based on the responses of the large model to the question requests in the training samples and the responses in the training samples to fine-tune the parameters of the large model.

7. A role-playing attack defense device based on security aspect, applied to the server of a large model, the large model is pre-adjusted using the method described in any one of claims 1 to 2, and the server is deployed with a security aspect program; the device comprises: A second data acquisition module is configured to acquire a question request initiated by a user to the large model through the server; A first identification module is configured to use the large model to identify whether there is role-playing content in the question request; A second identification module is configured to, in response to the large model identifying that the question request contains the role-playing content and outputting a reply including a security aspect program call instruction, intercept the reply of the large model to the question request by calling the security aspect program through the server, and detect the role-playing content; A first response module is configured to return a reply of the big model to the question request to the user in response to the big model identifying that the role-playing content does not exist in the question request; A second response module is configured to call the security aspect program to execute a role-playing attack defense strategy in response to detecting that the role-playing content is a malicious role-playing attack; And, in response to detecting that the role-playing content is a non-malicious role-playing attack, returning a reply of the large model to the question request to the user.

8. The device of claim 7, further comprising: The aspect program injection module is configured to obtain the application of the server and the aspect point of the application, and inject the security aspect program into the aspect point, so that the security aspect program is executed when the application is executed to the aspect point.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 2, or executes the method according to any one of claims 3 to 4.

10. An electronic device comprising: at least one memory for storing a program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 2, or execute the method according to any one of claims 3 to 4.

Images