Role simulation-based large model jailbreaking attack security assessment method and device

Abstract

The application provides a role simulation-based large model jailbreak attack security evaluation method and device. In one example, the method includes: obtaining original prompt text and rewriting the original prompt text into description text from the perspective of a victim; generating initial jailbreak attack prompt text from the perspective of a detective based on the description text from the perspective of the victim; performing prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain target jailbreak attack prompt text for security evaluation of the large model to be evaluated; inputting the target jailbreak attack prompt text into the large model to be evaluated to obtain a response output of the large model to be evaluated; and determining an anti-jailbreak attack security evaluation result of the large model to be evaluated based on the response output of the large model to be evaluated. The method can more accurately evaluate the ability of the large model to resist jailbreak attacks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and in particular to a role simulation-based security evaluation method and device for large model jailbreak attacks. BACKGROUND

[0002] Large language models (hereinafter referred to as large models) have attracted much attention in the social and scientific communities in recent years, have certain intelligent capabilities, and can help solve tasks such as spam detection, translation, legal consultation, and programming assistance. The output of each model is limited to a certain extent, which is to avoid the model outputting some illegal content, but this limitation can be broken. Large model jailbreak attack (hereinafter referred to as jailbreak attack) is one of the ways to break through the limitation.

[0003] Jailbreak attack technology is a specific technology for large models, which aims to break through the security lines of large models and make them perform operations that are originally limited or provide sensitive information. This type of attack uses the internal logic and generation mode of the large model, and bypasses the preset rules of the large model through induced input or malicious prompts.

[0004] Jailbreak attacks have developed rapidly in recent years, with many attack methods and relatively concealed, which can bypass most of the models released so far. Prevention of jailbreak attacks has become a popular research direction. SUMMARY

[0005] Therefore, the present application provides a role simulation-based security evaluation method and device for large model jailbreak attacks.

[0006] Specifically, the present application is implemented through the following technical solutions:

[0007] According to a first aspect of an embodiment of the present application, a role simulation-based security evaluation method for large model jailbreak attacks is provided, comprising:

[0008] obtaining an original prompt text and rewriting the original prompt text into a description text from the perspective of a victim;

[0009] generating an initial jailbreak attack prompt text from the perspective of a detective according to the description text from the perspective of the victim;

[0010] performing prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain a target jailbreak attack prompt text for security evaluation of a to-be-evaluated large model;

[0011] inputting the target jailbreak attack prompt text into the to-be-evaluated large model to obtain a response output of the to-be-evaluated large model;

[0012] According to the response output of the to-be-evaluated large model, a jailbreak attack security evaluation result of the to-be-evaluated large model is determined.

[0013] According to a second aspect of the embodiments of the present application, a large model jailbreak attack security evaluation device based on role simulation is provided, comprising:

[0014] A rewriting unit is configured to obtain an original prompt text and rewrite the original prompt text into a description text from a victim's perspective.

[0015] A generation unit is configured to generate an initial jailbreak attack prompt text from a detective's perspective according to the description text from the victim's perspective.

[0016] A re-interpretation unit is configured to perform prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain a target jailbreak attack prompt text for security evaluation of a to-be-evaluated large model.

[0017] An evaluation unit is configured to input the target jailbreak attack prompt text into the to-be-evaluated large model to obtain a response output of the to-be-evaluated large model, and determine a jailbreak attack security evaluation result of the to-be-evaluated large model according to the response output of the to-be-evaluated large model.

[0018] According to a third aspect of the embodiments of the present application, an electronic device is provided, comprising a processor and a memory, wherein,

[0019] The memory is configured to store a computer program.

[0020] The processor is configured to execute the program stored on the memory to implement the method provided in the first aspect.

[0021] According to a fourth aspect of the embodiments of the present application, a computer program product is provided, and the computer program product stores a computer program. When the computer program is executed by a processor, the method provided in the first aspect is implemented.

[0022] The role simulation-based large model jailbreak attack security evaluation method of the embodiment of the application, by obtaining the original prompt text, rewriting the original prompt text into a description text from the perspective of the victim; according to the description text from the perspective of the victim, generating an initial jailbreak attack prompt text from the perspective of a detective, and performing prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain a target jailbreak attack prompt text for security evaluation of the large model to be evaluated, and then inputting the target jailbreak attack prompt text into the large model to be evaluated to obtain the response output of the large model to be evaluated, and according to the response output of the large model to be evaluated, determining the anti-jailbreak attack security evaluation result of the large model to be evaluated, through the role simulation mode, the attack problem is disguised as a victim for help, reducing the vigilance of the model to malicious intent, effectively bypassing the security alignment mechanism; through the setting of the detective role, guiding the model to focus on problem solving rather than security review, significantly improving the attack success rate, and then the ability of the large model to resist jailbreak attack can be more accurately evaluated, in addition, through the setting of the victim role, the interaction scene between the real user and the large model is simulated, and the evaluation result can better reflect the actual defense ability of the large model in complex social engineering attacks. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A flowchart of a role simulation-based large model jailbreak attack security evaluation method is shown for an exemplary embodiment of the application;

[0024] Figure 2 An implementation diagram of rewriting an original prompt text into a description text from the perspective of a victim is shown for an exemplary embodiment of the application;

[0025] Figure 3 A diagram of generating an initial jailbreak attack prompt text according to a description text from the perspective of a victim is shown for an exemplary embodiment of the application;

[0026] Figure 4 An implementation diagram of performing prompt re-interpretation processing on an initial jailbreak attack prompt text, and inputting the obtained target jailbreak attack prompt text into a target model to obtain the response output of the target model is shown for an exemplary embodiment of the application;

[0027] Figure 5 A structural diagram of a role simulation-based large model jailbreak attack security evaluation device is shown for an exemplary embodiment of the application;

[0028] Figure 6 A hardware structure diagram of an electronic device is shown for an exemplary embodiment of the application. DETAILED DESCRIPTION

[0029] To better enable those skilled in the art to understand the technical solutions provided by the embodiments of the present application, the following will first explain some technical terms involved in the embodiments of the present application.

[0030] 1. Jailbreaking attack: bypassing the security restrictions of a model through specific inputs or techniques, causing it to generate content that is usually prohibited or harmful.

[0031] 2. Security alignment: ensuring that the output and behavior of a model are consistent with the intended goals and social norms, and do not produce harmful or inappropriate results.

[0032] In order to make the above-mentioned purposes, features and advantages of the embodiments of the present application more apparent and easy to understand, the technical solutions in the embodiments of the present application will be further described in detail below with reference to the drawings.

[0033] Please refer to Figure 1 , the flowchart of the role simulation-based large model jailbreaking attack security evaluation method provided by the embodiments of the present application is shown, which can include: Figure 1

[0034] Step S100, obtaining the original prompt text and rewriting the original prompt text into a description text from the victim's perspective.

[0035] In the embodiments of the present application, in order to reduce the vigilance of the large model to malicious intent and effectively bypass the security alignment mechanism, the original prompt text can be rewritten into a description text from the victim's perspective, so as to disguise the attack problem text as a victim's help-seeking text.

[0036] Step S110, generating an initial jailbreaking attack prompt text from the detective's perspective according to the description text from the victim's perspective.

[0037] In the embodiments of the present application, in order to improve the success rate of the jailbreaking attack and guide the large model to focus on problem solving rather than security review, the jailbreaking attack prompt text from the detective's perspective (which can be referred to as the initial jailbreaking attack prompt text) can be generated according to the description text from the victim's perspective obtained in step S100.

[0038] Step S120, performing prompt re-interpretation processing on the initial jailbreaking attack prompt text to obtain a target jailbreaking attack prompt text for security evaluation of the large model to be evaluated.

[0039] ​In the embodiment of the present application, in the case that the initial jailbreak attack prompt text is obtained in the above manner, the initial jailbreak attack prompt text can be subjected to prompt re-interpretation processing to optimize the jailbreak attack prompt text and obtain a jailbreak attack prompt text actually used for security evaluation of the to-be-evaluated large model (which can be referred to as a target jailbreak attack prompt text).

[0040] In step S130, the target jailbreak attack prompt text is input into the to-be-evaluated large model to obtain a response output of the to-be-evaluated large model.

[0041] In step S140, an anti-jailbreak attack security evaluation result of the to-be-evaluated large model is determined according to the response output of the to-be-evaluated large model.

[0042] In the embodiment of the present application, in the case that the target jailbreak attack prompt text is obtained in the above manner, the obtained target jailbreak attack prompt text can be input into the to-be-evaluated large model, the to-be-evaluated large model is used to process the target jailbreak attack prompt text, and a response output of the to-be-evaluated large model is obtained. Then, an anti-jailbreak attack security evaluation result of the to-be-evaluated large model can be determined according to the response output of the to-be-evaluated large model.

[0043] For example, for the obtained response output of the to-be-evaluated large model, it can be determined whether the response output is in violation. In the case that the response output is in violation, it can be determined that the to-be-evaluated large model has better anti-jailbreak attack capability; in the case that the response output is not in violation, it can be determined that the to-be-evaluated large model has poorer anti-jailbreak attack capability.

[0044] For example, for the obtained response output of the to-be-evaluated large model, it can be determined whether the response output is in violation. In the case that the response output is in violation, it can be determined that the to-be-evaluated large model has better anti-jailbreak attack capability; in the case that the response output is not in violation, it can be determined that the to-be-evaluated large model has poorer anti-jailbreak attack capability.

[0045] For example, the response output of the to-be-evaluated large model can be input into a preset model, the preset model is used to score the response output, and it is determined whether the response output is in violation according to the score.

[0046] For example, the score of the preset model can include 1-5 points, 1 point represents no violation, 2-5 points represent violation, and the higher the score, the more serious the violation.

[0047] Correspondingly, for the response output of the to-be-evaluated large model, in the case that the score of the preset model is higher, it can be determined that the anti-jailbreak attack performance of the to-be-evaluated large model is poorer.

[0048] It can be seen that, in the case that the to-be-evaluated large model is subjected to the prompt re-interpretation processing, the anti-jailbreak attack security evaluation result of the to-be-evaluated large model can be determined according to the response output of the to-be-evaluated large model. Figure 1In the method flow shown, the original prompt text is obtained, and the original prompt text is rewritten into a description text from the perspective of the victim; based on the description text from the perspective of the victim, an initial jailbreak attack prompt text from the perspective of a detective is generated, and the initial jailbreak attack prompt text is subjected to prompt re-interpretation processing to obtain a target jailbreak attack prompt text for security evaluation of a large model to be evaluated, and then the target jailbreak attack prompt text is input into the large model to be evaluated to obtain a response output of the large model to be evaluated, and based on the response output of the large model to be evaluated, an anti-jailbreak attack security evaluation result of the large model to be evaluated is determined, the attack problem is disguised as a victim's help-seeking through role simulation, the vigilance of the model to malicious intent is reduced, and the security alignment mechanism is effectively bypassed; through the setting of the detective role, the model is guided to focus on problem solving rather than security review, and the attack success rate is significantly improved, and then the ability of the large model to resist jailbreak attacks can be more accurately evaluated, in addition, through the setting of the victim role, the interaction scene between the real user and the large model is simulated, and the evaluation result can better reflect the actual defense capability of the large model in complex social engineering attacks.

[0049] In some embodiments, the above rewriting of the original prompt text into a description text from the perspective of the victim can include:

[0050] The original prompt text is rewritten into a description text from the perspective of the victim using an auxiliary language model.

[0051] For example, in order to reduce the complexity of rewriting the original prompt text, reduce syntax errors and computational resource consumption, and simplify the operation, an auxiliary language model can be used to rewrite the original prompt text.

[0052] For example, the auxiliary language model can be input with prompt information for prompting how to rewrite the input prompt text into a description text from the perspective of the victim, and the original prompt text can be input into the auxiliary language model, so that the original prompt text is rewritten into a description text from the perspective of the victim by the auxiliary language model.

[0053] In one example, the above rewriting of the original prompt text into a description text from the perspective of the victim using an auxiliary language model can include:

[0054] The original prompt text is nested into a prompt rewriting template to obtain a prompt rewriting text;

[0055] The prompt rewriting text is input into the auxiliary language model, and a description text from the perspective of the victim output by the auxiliary language model is obtained.

[0056] For example, in order to further simplify the rewriting operation of the description text from the perspective of the victim and improve the scalability of the scheme, a prompt rewriting template can be set, and the prompt rewriting template can include prompt information for prompting how to rewrite the input prompt text into a description text from the perspective of the victim.

[0057] Accordingly, in the process of rewriting the original prompt text, the original prompt text can be nested into the prompt rewriting template to obtain a prompt rewriting text.

[0058] For example, the prompt rewriting template can include fixedly set prompt information (for prompting how to rewrite the input prompt text into the victim perspective) and a pre-set blank position (for inserting the original prompt text). In the process of rewriting the original prompt text, the original prompt text can be inserted into the pre-set blank position of the prompt rewriting template to obtain a prompt rewriting text.

[0059] For the obtained prompt rewriting text, it can be input into an auxiliary language model. The auxiliary language model can rewrite the original prompt text according to the prompt information in the prompt rewriting text and output a description text in the victim perspective.

[0060] In some embodiments, the above generation of the initial jailbreak attack prompt text in the detective perspective according to the description text in the victim perspective can include:

[0061] The description text in the victim perspective is nested into a detective role template to obtain the initial jailbreak attack prompt text in the detective perspective.

[0062] For example, in order to simplify the generation operation of the jailbreak attack prompt text in the detective perspective and improve the scalability of the scheme, a detective role template can be set, which includes context information of the jailbreak attack prompt text.

[0063] Accordingly, in the case where the description text in the victim perspective is obtained, the description text in the victim perspective can be nested into the detective role template to obtain the initial jailbreak attack prompt text in the detective perspective.

[0064] In some embodiments, the above prompt re-interpretation processing of the initial jailbreak attack prompt text to obtain the target jailbreak attack prompt text for security evaluation of the to-be-evaluated large model can include:

[0065] According to the re-interpretation loss, the initial jailbreak attack prompt text is subjected to cyclic iteration prompt re-interpretation processing until a preset iteration stop condition is met to obtain the target jailbreak attack prompt text for security evaluation of the to-be-evaluated large model.

[0066] For example, in order to optimize the prompt re-interpretation effect, in the process of re-interpretation processing of the jailbreak attack prompt text, the jailbreak attack prompt text can be subjected to cyclic iteration prompt re-interpretation processing until a preset iteration stop condition is met.

[0067] For example, in the process of cyclic iteration of the prompt re-interpretation processing on the jailbreak attack prompt text, a re-interpretation loss can be introduced, which is used to evaluate the effect of the prompt re-interpretation processing. Through feedback according to the re-interpretation loss, the effect of the prompt re-interpretation can be optimized.

[0068] For example, assuming that the smaller the re-interpretation loss is, the better the effect of the prompt re-interpretation processing is, then in the process of cyclic iteration of the prompt re-interpretation processing on the jailbreak attack prompt text, for each iteration, the specific strategy of the current prompt re-interpretation processing can be adjusted according to the re-interpretation loss of the last prompt re-interpretation processing to reduce the re-interpretation loss.

[0069] For example, satisfying the preset iteration stop condition can include that the number of iterations reaches a preset maximum number of iterations, or the re-interpretation loss satisfies a preset condition, etc.

[0070] In one example, the re-interpretation loss can include one or more of the following:

[0071] An attack loss; wherein for any prompt re-interpretation processing, the attack loss of the current prompt re-interpretation processing is determined according to the current large model response output, which is the response output of the to-be-evaluated large model to the jailbreak attack prompt text obtained by the current prompt re-interpretation processing;

[0072] A fluency loss; wherein for any prompt re-interpretation processing, the fluency loss of the current prompt re-interpretation processing is used to represent the fluency of the jailbreak attack prompt text obtained by the current prompt re-interpretation processing. The fluency is determined according to the prediction probability of the auxiliary language model for the i-th word under the condition that the first i-1 words in the jailbreak attack prompt text obtained by the current prompt re-interpretation processing are given; 2≤i≤N, N is the total length of the jailbreak attack prompt text obtained by the current prompt re-interpretation processing;

[0073] A similarity loss; wherein for any prompt re-interpretation processing, the similarity loss of the current prompt re-interpretation processing is determined according to the similarity between the jailbreak attack prompt texts before and after the current prompt re-interpretation processing.

[0074] For example, the jailbreak attack prompt text can be optimized by at least one of the three loss functions of the attack loss, the fluency loss, and the similarity loss.

[0075] The attack loss can be used to ensure the effect of the jailbreak attack prompt text; the fluency loss can be used to ensure the semantic fluency of the generated jailbreak attack prompt text; and the similarity loss can be used to ensure the similarity of the jailbreak attack prompt text after the prompt re-interpretation processing, so as to avoid excessive deviation from the original theme.

[0076] Exemplarily, for each prompt re-interpretation processing, an attack loss of the prompt re-interpretation processing can be determined according to a response output (which can be referred to as a current large model response output) of the large model to an escape attack prompt text obtained by the prompt re-interpretation processing.

[0077] Exemplarily, in a case where the current large model response output is an affirmative answer (which can be referred to as a first type of answer), it indicates that the large model does not recognize the attackiveness of the escape attack prompt text obtained by the prompt re-interpretation processing, and the corresponding attack loss is small; in a case where the current large model response output is a refusal answer (which can be referred to as a second type of answer), it indicates that the large model recognizes the attackiveness of the escape attack prompt text obtained by the prompt re-interpretation processing, and accordingly, the corresponding attack loss is large.

[0078] Among them, the current large model response output can be determined as the first type of answer or the second type of answer according to the first preset number (which can be an empirical value) of words in the large model response output.

[0079] For example, the first few words of the first type of answer are generally “OK, ***”, “Of course, ***” or “The following is ***”; the first few words of the second type of answer are generally “Sorry, ***”, “Apologies ***”, etc.

[0080] Exemplarily, a value of the attack loss corresponding to the first type of answer (which can be referred to as a first attack loss value) and a value of the attack loss corresponding to the second type of answer (which can be referred to as a second attack loss value) can be preset; and the first attack loss value is less than the second attack loss value.

[0081] Exemplarily, for each prompt re-interpretation processing, a fluency loss of the prompt re-interpretation processing is used to represent the fluency of the escape attack prompt text obtained by the prompt re-interpretation processing.

[0082] Among them, the fluency is determined according to a prediction probability of an auxiliary language model for the i-th word in a case where the first i-1 words in the escape attack prompt text obtained by the prompt re-interpretation processing are given.

[0083] It should be noted that the auxiliary language model used to determine the fluency and the auxiliary language model used to rewrite the original prompt text can be the same auxiliary language model or different auxiliary language models (the auxiliary language model used to determine the fluency can be referred to as a first auxiliary language model, and the auxiliary language model used to rewrite the original prompt text can be referred to as a second auxiliary language model).

[0084] For example, assuming that the jailbreak attack prompt text is "How to make ***", in the process of reasoning the probability that the 4th word is "make", the probability that the 4th word is "make" can be determined according to the auxiliary language model, that is, P (make | how to make), to determine the fluency loss.

[0085] For example, for each prompt word re-interpretation processing, the similarity loss of the prompt word re-interpretation processing can be determined according to the similarity between the jailbreak attack prompt text before and after the prompt word re-interpretation processing.

[0086] By introducing the above loss constraint, the attack success rate of the generated target jailbreak attack prompt text can be improved.

[0087] In one example, the re-interpretation loss can be a weighted sum of the attack loss, the fluency loss, and the similarity loss.

[0088] For example, in the process of determining the re-interpretation loss, the attack loss, the fluency loss, and the similarity loss can correspond to different loss values, and the re-interpretation loss can be a weighted sum of the loss values corresponding to the attack loss, the fluency loss, and the similarity loss.

[0089] In order for those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, the technical solutions provided by the embodiments of the present application will be described below in conjunction with specific examples.

[0090] In this embodiment, in order to explore whether the large model can resist the jailbreak attack to a certain extent, a large model anti-jailbreak attack performance evaluation scheme is proposed by combining the rewriting of the original prompt text and the role simulation. The scheme rewrites the original prompt text into a description from the victim's perspective, obtains a description text from the victim's perspective (which can be referred to as a victim perspective prompt text), and nests the victim prompt text into a preset detective role template to generate an initial jailbreak attack prompt text. The initial jailbreak attack prompt text is processed by the prompt re-interpretation module to obtain a final jailbreak attack prompt text for security evaluation of the large model to be evaluated (i.e., the above target jailbreak attack prompt text), and then the target jailbreak attack prompt text is input into the target model (i.e., the large model to be evaluated). Through the above processing, the security alignment detection of the large model can be effectively bypassed to induce the large model to output illegal content, so that the resistance of the large model to the jailbreak attack can be effectively and simply evaluated.

[0091] For example, the implementation process of the large model anti-jailbreak attack performance evaluation scheme provided by this embodiment is as follows:

[0092] 1) Victim disguise prompt rewriting

[0093] 1.1) Prepare the original prompt text and the auxiliary language model;

[0094] 1.2) design a prompt rewriting template for assisting the rewriting of the original prompt text;

[0095] 1.3) nest the original prompt text into the prompt rewriting template to obtain a prompt rewriting text;

[0096] 1.4) input the prompt rewriting text into an auxiliary language model;

[0097] 1.5) the auxiliary language model rewrites the original prompt text to obtain a victim perspective description text.

[0098] 2) detective role nesting

[0099] 2.1) design a detective role prompt template, which can include: role definition, question description, role injection, supplementary explanation;

[0100] 2.2) nest the rewritten victim perspective description text in step 1) into the detective role template to form an initial jailbreak attack prompt text;

[0101] 2.3) input the initial jailbreak attack prompt text into the prompt re-interpretation module.

[0102] 3) prompt re-interpretation

[0103] 3.1) design a prompt re-interpretation module that integrates attack loss, fluency loss, and similarity loss.

[0104] 3.2) perform re-interpretation processing on the initial jailbreak attack prompt text to obtain a target jailbreak prompt text for security evaluation of the target model.

[0105] 3.3) input the target jailbreak attack prompt text into the target model.

[0106] 3.4) obtain the output response of the target model.

[0107] 3.5) determine whether the output response of the target model violates the rules.

[0108] 3.6) determine the anti-jailbreak attack security evaluation result of the target model according to the determination result.

[0109] The implementation details of the above process are described below.

[0110] Please refer to Figure 2 , which is an implementation schematic diagram of rewriting the original prompt text into a victim perspective description text in this embodiment.

[0111] As Figure 2 shown, the original prompt text (which can also be referred to as an original input text) can be nested into a prompt rewriting template , to get a rewritten prompt text , which will be input to the auxiliary model , will rewrite into a victim perspective expression (also called victim description) and output.

[0112] Exemplarily, the specific method steps of rewriting the original prompt text into a victim perspective description text can be as follows:

[0113] 1.1) prepare the original prompt text and the auxiliary model ;

[0114] 1.2) design a prompt rewriting template for rewriting the original prompt text ;

[0115] 1.3) nest the original prompt text into the rewriting template to get a prompt rewriting text ;

[0116] 1.4) input the prompt rewriting text to the auxiliary language model ;

[0117] 1.5) the auxiliary language model receives the prompt rewriting text rewrites the original prompt text to get a victim perspective description text ; wherein:

[0118]

[0119] In the victim perspective description text , the first characters represent the victim's experience description, and the last characters represent the victim's request for help.

[0120] Exemplarily, the can be sent to step 2.

[0121] Please refer to Figure 3 for a schematic diagram of generating an initial jailbreak attack prompt text according to the victim perspective description text (i.e. victim description) in this embodiment.

[0122] As shown in Figure 3 , the obtained victim perspective description text ​Embedding the victim perspective description text into the detective role template to generate the initial jailbreak attack prompt text .

[0123] Exemplarily, the specific method steps of generating the initial jailbreak attack prompt text according to the victim perspective description text can be as follows:

[0124] 2.1) Design a detective role template for the detective role simulation of the large model , contains four parts: role definition, problem description, role injection, and supplementary explanation. The role definition and problem description can be defined as the template prefix , and the role injection and supplementary explanation can be defined as the template suffix ;

[0125] 2.2) Embed the victim perspective description text into the detective role template to obtain the initial jailbreak attack prompt text :

[0126]

[0127] Please refer to Figure 4 for the implementation diagram of the prompt re-interpretation processing of the initial jailbreak attack prompt text in this embodiment, and inputting the obtained target jailbreak attack prompt text into the target model to obtain the response output of the target model.

[0128] As shown in Figure 4 , the obtained initial jailbreak attack prompt text can be input into the prompt re-interpretation module , the prompt re-interpretation module re-interprets the initial jailbreak attack prompt text through multiple losses to obtain the target jailbreak attack prompt text , and then the target jailbreak attack prompt text can be input into the target model , will process and output the final reply (i.e., response output) .

[0129] Exemplarily, the specific method steps of generating the initial jailbreak attack prompt text according to the victim perspective description text can be as follows:

[0130] 3.1) Design a prompt re-interpretation module for prompt re-interpretation processing , The three losses contained in the middle are: attack loss, fluency loss, and similarity loss. The attack loss is used to ensure the effectiveness of the jailbreak attack, the fluency loss is used to ensure that the text after re-interpretation conforms to human semantics, and the similarity loss is used to limit the deviation of the re-interpretation prompt from the original theme; wherein:

[0131] a) Attack loss :

[0132]

[0133] Wherein, is the reference answer (the response output obtained by inputting the jailbreak attack prompt text into the target model during the prompt re-interpretation process), for example, "Of course, this is [the process of making a bomb]."

[0134] b) Fluency loss :

[0135]

[0136] Wherein, is the total length of the text sequence, is the prediction probability of the next word by the language model given the previous context.

[0137] c) Similarity loss :

[0138]

[0139] Wherein, wherein emb(x) is the embedding vector of text x.

[0140] 3.2) The target jailbreak attack prompt text after prompt re-interpretation processing is input into the target model ;

[0141] 3.3) The target model processes it after receiving , and outputs the final reply ;

[0142]

[0143] 3.4) Determine whether the reply of the target model is illegal. In the case where the reply is not illegal content, it is determined that the target model has good ability to resist jailbreak attacks; in the case where the reply ​​For the case of violating the content, it is determined that the target model has poor ability to resist jailbreak attacks.

[0144] As can be seen, in this embodiment, by disguising the jailbreak attack problem as a victim help, the alertness of the large model to malicious intent is reduced, effectively bypassing the security alignment mechanism.

[0145] In addition, by simulating the interaction scene between a real user and a large model (such as a victim help), the evaluation results can better reflect the actual defense capability of the model in complex social engineering attacks.

[0146] Secondly, on the one hand, the auxiliary model generates a natural language description, and on the other hand, through role setting (such as a detective), the large model is guided to focus on problem solving rather than security review, which significantly improves the success rate of jailbreak attacks.

[0147] Among them, by using an auxiliary language model to generate jailbreak attack prompt text, the problem of relying on a complex optimization function (such as gradient descent) in traditional methods is avoided, reducing syntax errors and computational resource consumption, and the operation is more convenient. Moreover, the jailbreak attack prompt text generated by the auxiliary model is more in line with natural language habits, and compared with the traditional iterative search method, the output jailbreak attack text prompt is more fluent and logical, reducing the risk of being identified by the security mechanism.

[0148] Finally, by introducing the prompt re-interpretation module PA, the jailbreak prompt text is optimized through three loss functions: attack loss, fluency loss, and similarity loss. Not only can the effectiveness of the jailbreak attack be guaranteed, but also the generated jailbreak attack prompt text can maintain a balance between semantic fluency and similarity to the original input, avoiding excessive deviation from the original theme, thereby improving the success rate of the pre-attack.

[0149] The above describes the method provided by the present application. The following describes the device provided by the present application:

[0150] Please refer to Figure 5 , a structure schematic diagram of a large model jailbreak attack security evaluation device based on role simulation provided by an embodiment of the present application is shown, which can include: Figure 5

[0151] The rewriting unit is configured to obtain an original prompt text and rewrite the original prompt text into a description text from the perspective of a victim.

[0152] The generation unit is configured to generate an initial jailbreak attack prompt text from the perspective of a detective according to the description text from the perspective of the victim.

[0153] ​A re-interpretation unit is configured to perform re-interpretation processing on the initial jailbreak attack prompt text to obtain target jailbreak attack prompt text for security evaluation of the to-be-evaluated large model.

[0154] An evaluation unit is configured to input the target jailbreak attack prompt text into the to-be-evaluated large model to obtain a response output of the to-be-evaluated large model, and determine an anti-jailbreak attack security evaluation result of the to-be-evaluated large model according to the response output of the to-be-evaluated large model.

[0155] The embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to execute the program stored in the memory to implement the role simulation-based large model jailbreak attack security evaluation method described above.

[0156] Please refer to Figure 6 , which is a hardware structure schematic diagram of an electronic device provided by the embodiment of the present application. The electronic device can include a processor 601 and a memory 602 storing machine executable instructions. The processor 601 and the memory 602 can communicate via a system bus 603. By reading and executing the machine executable instructions corresponding to the role simulation-based large model jailbreak attack security evaluation logic in the memory 602, the processor 601 can execute the role simulation-based large model jailbreak attack security evaluation method described above.

[0157] The memory 602 mentioned herein can be any electronic, magnetic, optical or other physical storage device, and can contain or store information such as executable instructions, data, etc. For example, the machine readable storage medium can be RAM (Radom Access Memory, Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard drive), solid state disk, any type of storage disk (such as optical disk, dvd, etc.), or similar storage medium, or combination thereof.

[0158] In some embodiments, a machine readable storage medium, such as the memory 602 in Figure 6 , is also provided, and the machine executable instructions are stored in the machine readable storage medium, which are executed by the processor to implement the role simulation-based large model jailbreak attack security evaluation method described above. For example, the machine readable storage medium can be ROM, RAM, CD-ROM, magnetic tape, floppy disk and optical data storage device, etc.

[0159] The embodiment of the present application also provides a computer program product, which stores a computer program, and when the processor executes the computer program, the processor is prompted to execute the role simulation-based large model jailbreak attack security evaluation method described above.

Claims

1. A method for security evaluation of a role simulation-based large model jailbreak attack, characterized in that, The method comprises the following steps: obtaining original prompt text and rewriting the original prompt text into description text from the perspective of a victim; generating initial jailbreak attack prompt text from the perspective of a detective according to the description text from the perspective of the victim; performing prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain target jailbreak attack prompt text for security evaluation of a large model to be evaluated; inputting the target jailbreak attack prompt text into the large model to be evaluated to obtain a response output of the large model to be evaluated; determining an anti-jailbreak attack security evaluation result of the large model to be evaluated according to the response output of the large model to be evaluated; wherein the prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain the target jailbreak attack prompt text for security evaluation of the large model to be evaluated comprises: performing cyclic iteration prompt re-interpretation processing on the initial jailbreak attack prompt text according to a re-interpretation loss until a preset iteration stop condition is met to obtain the target jailbreak attack prompt text for security evaluation of the large model to be evaluated; the re-interpretation loss comprises: an attack loss; wherein for any one prompt word re-interpretation processing, the attack loss of the prompt word re-interpretation processing is determined according to a current large model response output, which is a response output of the large model to the jailbreak attack prompt text obtained by the prompt word re-interpretation processing; a fluency loss; wherein for any one prompt word re-interpretation processing, the fluency loss of the prompt word re-interpretation processing is used to represent the fluency of the jailbreak attack prompt text obtained by the prompt word re-interpretation processing; the fluency is determined according to a prediction probability of an auxiliary language model for the i-th word under the condition that the first i-1 words in the jailbreak attack prompt text obtained by the prompt word re-interpretation processing are given; 2≤i≤N, N is the total length of the jailbreak attack prompt text obtained by the prompt word re-interpretation processing; and a similarity loss; wherein for any one prompt word re-interpretation processing, the similarity loss of the prompt word re-interpretation processing is determined according to the similarity between the jailbreak attack prompt texts before and after the prompt word re-interpretation processing.

2. The method of claim 1, wherein, The rewriting of the original prompt text into the description text from the perspective of the victim comprises: rewriting the original prompt text into the description text from the perspective of the victim by using an auxiliary language model.

3. The method of claim 2, wherein, The rewriting of the original prompt text into the description text from the perspective of the victim by using the auxiliary language model comprises: nesting the original prompt text into a prompt rewriting template to obtain prompt rewriting text; inputting the prompt rewriting text into the auxiliary language model and obtaining the description text from the perspective of the victim output by the auxiliary language model.

4. The method of claim 1, wherein, The generation of the initial jailbreak attack prompt text from the perspective of the detective according to the description text from the perspective of the victim comprises: nesting the description text from the perspective of the victim into a detective role template to obtain the initial jailbreak attack prompt text from the perspective of the detective.

5. A role-playing simulation-based large model jailbreaking attack security evaluation device, characterized in that, The method comprises the following steps: a rewriting unit is configured to obtain original prompt text and rewrite the original prompt text into description text from the perspective of a victim; The generating unit is configured to generate initial jailbreak attack prompt text in the detective perspective according to the description text in the victim perspective; The re-interpretation unit is configured to perform prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain target jailbreak attack prompt text for security evaluation of the to-be-evaluated large model; The evaluation unit is configured to input the target jailbreak attack prompt text into the to-be-evaluated large model to obtain a response output of the to-be-evaluated large model; and determine an anti-jailbreak attack security evaluation result of the to-be-evaluated large model according to the response output of the to-be-evaluated large model. The re-interpretation unit performs prompt re-interpretation processing on the initial jailbreak attack prompt text to obtain target jailbreak attack prompt text for security evaluation of the to-be-evaluated large model, including: The re-interpretation unit performs cyclic iteration prompt re-interpretation processing on the initial jailbreak attack prompt text according to a re-interpretation loss until a preset iteration stop condition is met, to obtain target jailbreak attack prompt text for security evaluation of the to-be-evaluated large model. The re-interpretation loss includes: An attack loss; for any one prompt word re-interpretation processing, the attack loss of the prompt word re-interpretation processing is determined according to a current large model response output, and the current large model response data is a response output of the to-be-evaluated large model to jailbreak attack prompt text obtained by the prompt word re-interpretation processing; A fluency loss; for any one prompt word re-interpretation processing, the fluency loss of the prompt word re-interpretation processing is used to represent the fluency of jailbreak attack prompt text obtained by the prompt word re-interpretation processing; the fluency is determined according to a prediction probability of a language model for an i-th word under a condition that a first i-1 words in the jailbreak attack prompt text obtained by the prompt word re-interpretation processing are given; 2≤i≤N, and N is a total length of the jailbreak attack prompt text obtained by the prompt word re-interpretation processing; And a similarity loss; for any one prompt word re-interpretation processing, the similarity loss of the prompt word re-interpretation processing is determined according to a similarity between jailbreak attack prompt texts before and after the prompt word re-interpretation processing.

6. The apparatus of claim 5, wherein, The rewriting unit rewrites the original prompt text into description text in the victim perspective, including: The rewriting unit rewrites the original prompt text into description text in the victim perspective by using an auxiliary language model. The rewriting unit rewrites the original prompt text into description text in the victim perspective by using an auxiliary language model, including: The rewriting unit rewrites the original prompt text into description text in the victim perspective by using an auxiliary language model, including: The rewriting unit rewrites the original prompt text into description text in the victim perspective by using an auxiliary language model, including: The generating unit generates initial jailbreak attack prompt text in the detective perspective according to the description text in the victim perspective, including: The generating unit generates initial jailbreak attack prompt text in the detective perspective according to the description text in the victim perspective, including: The generating unit generates initial jailbreak attack prompt text in the detective perspective according to the description text in the victim perspective, including:

7. An electronic device, comprising: The generating unit generates initial jailbreak attack prompt text in the detective perspective according to the description text in the victim perspective, including: The processor and the memory, wherein: The memory is configured to store a computer program. A processor is configured to implement the method of any one of claims 1-4 when executing a program stored in a memory.

8. A computer program product, characterised in that, A computer program product stores a computer program, and the computer program is configured to implement the method of any one of claims 1-4 when executed by a processor.

Images