Big model-based jailbreak prompt generation method, electronic device, storage medium and computer program product

Abstract

This application provides a jailbreak hint generation method, electronic device, storage medium, and computer program product based on a large model. The jailbreak hint generation method includes: acquiring multiple initial jailbreak hints; encoding each initial jailbreak hint into a vector, and constructing an initial simplex based on the vector representation of each initial jailbreak hint, wherein the initial simplex contains multiple vertices, each vertex corresponding to a vector representation of an initial jailbreak hint; acquiring the objective function value of each vertex, the objective function value representing the harmfulness of the initial jailbreak hint; iteratively performing geometric transformations on the initial simplex based on the objective function value of each vertex and a simplex search strategy until a preset iteration termination condition is reached, obtaining an updated simplex; and obtaining the target jailbreak hint based on the updated simplex. This application can improve the efficiency of jailbreak hint generation and reduce computational costs.

Description

Technical Field

[0001] This application relates to the field of AI, and more specifically, to a method for generating jailbreak prompts based on a large model, an electronic device, a storage medium, and a computer program product. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models have been widely used in text generation, intelligent question answering, and content creation. To prevent large models from generating harmful, illegal, or inappropriate content, developers typically set up security protection mechanisms for the models, including modules for sensitive information filtering, identification of illegal intent, and content moderation. These mechanisms together constitute the "security boundary" of the large model. However, the security protection mechanisms of large models are not absolutely reliable. Attackers or testers can bypass security protections by constructing specific prompt text (i.e., "jailbreak prompts") to induce the model to generate illegal content. This process is called "large model jailbreaking." The existence of jailbreak prompts exposes security vulnerabilities in large models, which may be maliciously exploited to spread harmful information. Therefore, detecting and fixing these vulnerabilities in advance is crucial for the secure application of large models.

[0003] Currently, existing jailbreak hint generation methods require a large amount of labeled data for training, and the model training process is complex and time-consuming. The generated hint text is highly random, making it difficult to accurately locate security vulnerabilities in large models. Summary of the Invention

[0004] The purpose of this application is to provide a jailbreak prompt generation method, electronic device, storage medium, and computer program product based on a large model, so as to improve the jailbreak prompt generation efficiency and reduce the computational cost.

[0005] In a first aspect, the present invention provides a jailbreak prompt generation method based on a large model, comprising:

[0006] Get multiple initial jailbreak prompts;

[0007] The multiple initial jailbreak prompts are encoded into vectors, and an initial simplex is constructed based on the vector representation of each initial jailbreak prompt. The initial simplex contains multiple vertices, and each vertex corresponds to a vector representation of an initial jailbreak prompt.

[0008] Obtain the objective function value for each vertex, the objective function value representing the harmfulness of the initial jailbreak prompt;

[0009] The initial simplex is geometrically transformed iteratively based on the objective function value and simplex search strategy of each vertex until a preset iteration termination condition is met, and the updated simplex is obtained.

[0010] The target jailbreak prompt is obtained based on the updated simplex.

[0011] The method of the first aspect of this application constructs an initial simplex based on the initial jailbreak prompt. Then, it processes the initial simplex using a simplex search strategy to obtain an updated simplex, thereby obtaining an optimized initial jailbreak prompt based on the updated simplex—that is, a set of prompts that more closely approximates the optimal jailbreak prompt. Compared with existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the prompt text through iteration, avoiding the tedious process of manually constructing rules and training models, thus significantly improving the efficiency of jailbreak prompt generation. Furthermore, it eliminates the need for model training, reducing computational costs.

[0012] In an optional implementation, the geometric transformation of the initial simplex based on the objective function value of each vertex and the simplex search strategy includes:

[0013] Based on the objective function value of each vertex, all vertices are sorted in descending order, and the first vertex, second vertex, third vertex, and fourth vertex are determined based on the sorting result. The first vertex is the vertex with the highest objective function value, the second vertex is the vertex with the second highest objective function value, the third vertex is the vertex with the second lowest objective function value, and the fourth vertex is the vertex with the lowest objective function value.

[0014] The centroid is determined based on all vertices except the fourth vertex, wherein all vertices except the fourth vertex include the first vertex, the second vertex, and the third vertex;

[0015] Obtain the reflection coefficient, and determine the reflection point based on the reflection coefficient, the centroid, and the fourth vertex;

[0016] The jailbreak prompt for the reflected point is determined;

[0017] Determine the target function value of the reflection point based on the jailbreak hints at the reflection point;

[0018] When the objective function value of the reflection point is greater than or equal to the objective function value of the fourth vertex and less than or equal to the objective function value of the first vertex, the initial jailbreak prompt of the fourth vertex is replaced with the jailbreak prompt of the reflection point.

[0019] This optional implementation can perform a geometric transformation on the initial simplex based on the reflection point. Specifically, when the objective function value of the reflection point is greater than or equal to the objective function value of the fourth vertex and less than or equal to the objective function value of the first vertex, it means that a better adversarial cue than the current worst point has been found. The worst vertex is replaced with the reflection point, the simplex is updated, and through iteration, a better adversarial cue can be generated.

[0020] In an optional implementation, the method further includes:

[0021] When the objective function value at the reflection point is greater than the objective function value at the first vertex, the expansion coefficient is obtained;

[0022] The expansion point is determined based on the expansion coefficient, the centroid, and the reflection point;

[0023] The jailbreak prompt for the expansion point is confirmed;

[0024] Determine the target function value of the extension point based on the jailbreak prompts of the extension point;

[0025] If the objective function value at the expansion point is greater than the objective function value at the reflection point, then the initial jailbreak prompt at the fourth vertex is replaced with the jailbreak prompt at the expansion point.

[0026] In this optional implementation, if the objective function value at the expansion point is greater than the objective function value at the reflection point, then the jailbreak prompt at the expansion point replaces the initial jailbreak prompt at the fourth vertex, thereby generating a better adversarial prompt.

[0027] In an optional implementation, the method further includes:

[0028] When the objective function value at the reflection point is less than the objective function value at the fourth vertex, the contraction coefficient is obtained;

[0029] The contraction point is determined based on the contraction coefficient, the centroid, and the four vertices;

[0030] The jailbreak prompt at the contraction point is confirmed;

[0031] Determine the objective function value of the contraction point based on the jailbreak hints at the contraction point;

[0032] When the objective function value at the contraction point is greater than the objective function value at the fourth vertex, the initial jailbreak prompt at the fourth vertex is replaced by the jailbreak prompt at the contraction point.

[0033] In this optional implementation, when the objective function value of the reflection point is less than the objective function value of the fourth vertex, and the objective function value of the contraction point is greater than the objective function value of the fourth vertex, the initial jailbreak prompt of the fourth vertex can be replaced with the jailbreak prompt of the contraction point, thereby generating a better adversarial prompt.

[0034] In an optional implementation, the method further includes:

[0035] When the objective function value of the jailbreak prompt at the contraction point is less than or equal to the objective function value of the fourth vertex, modify the jailbreak prompts of all vertices except the first vertex so that the jailbreak prompts of all vertices except the first vertex are similar to the jailbreak prompt of the first vertex.

[0036] This optional implementation can modify the jailbreak prompts of all vertices except the first vertex when the target function value of the jailbreak prompt at the contraction point is less than the target function value of the fourth vertex, so that the jailbreak prompts of all vertices except the first vertex are similar to the jailbreak prompt of the first vertex.

[0037] In an optional implementation, the preset iteration termination condition includes reaching the maximum number of iterations, or the improvement of the objective function value in a number of consecutive iterations being less than a preset threshold.

[0038] This optional implementation can terminate the iteration when the maximum number of iterations is reached or the improvement of the objective function value in several consecutive iterations is less than a preset threshold, thus avoiding excessive deformation of the simplex.

[0039] In an optional implementation, encoding the plurality of initial jailbreak prompts into vectors includes:

[0040] Each initial jailbreak prompt is semantically vectorized using a text encoder to obtain a vector representation of the initial jailbreak prompt.

[0041] This optional implementation can utilize a text encoder to semantically vectorize each of the initial jailbreak prompts, thereby obtaining a vector representation of the initial jailbreak prompts.

[0042] In an optional implementation, obtaining the objective function value for each vertex includes:

[0043] Input the initial jailbreak prompt corresponding to the vertex into the target large model to obtain the response text of the target large model;

[0044] The response text is scored based on a pre-trained harmfulness classification model, and the score is used as the objective function value of the vertex.

[0045] This optional implementation can input the initial jailbreak prompt corresponding to the vertex into the target large model to obtain the response text of the target large model, and then score the response text based on the pre-trained harmfulness classification model, with the obtained score serving as the objective function value of the vertex.

[0046] In an optional implementation, the jailbreak prompt for determining the reflection point includes:

[0047] Determine the semantic offset vector between the first vertex and the reflection point;

[0048] Based on the semantic offset vector, perform at least one of the following operations on the first vertex jailbreak prompt: keyword replacement, modifier addition / deletion, or sentence structure adjustment, to generate multiple candidate jailbreak prompts;

[0049] Select the candidate jailbreak hint that is closest to the reflection point vector from the plurality of candidate jailbreak hints as the jailbreak hint for the reflection point.

[0050] This optional implementation can generate candidate jailbreak prompts using at least one of the following operations: keyword replacement, addition or deletion of modifiers, or adjustment of sentence structure.

[0051] Secondly, the present invention provides a jailbreak prompt generation device based on a large model, comprising:

[0052] The first acquisition module is used to acquire multiple initial jailbreak prompts;

[0053] The encoding module is used to encode the plurality of initial jailbreak prompts into vectors respectively, and to construct an initial simplex based on the vector representation of each initial jailbreak prompt, wherein the initial simplex contains a plurality of vertices, and each vertex corresponds to a vector representation of the initial jailbreak prompt;

[0054] The second acquisition module is used to acquire the objective function value of each vertex, wherein the objective function value characterizes the degree of harm of the initial jailbreak prompt;

[0055] The calculation module is used to iteratively perform geometric transformations on the initial simplex based on the objective function value and simplex search strategy of each vertex until a preset iteration termination condition is reached, and then obtain the updated simplex.

[0056] A generation module is used to obtain the target jailbreak prompt based on the updated simplex.

[0057] The apparatus of the second aspect of this application can construct an initial simplex based on an initial jailbreak prompt. Then, it processes the initial simplex using a simplex search strategy to obtain an updated simplex, thereby obtaining an optimized initial jailbreak prompt based on the updated simplex, i.e., a prompt set that more closely approximates the optimal jailbreak prompt. Compared with existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the prompt text through iteration, avoiding the tedious process of manually constructing rules and training models, thus significantly improving the efficiency of jailbreak prompt generation. Furthermore, it eliminates the need for model training, reducing computational costs.

[0058] Thirdly, the present invention provides an electronic device, comprising:

[0059] Processor; and

[0060] The memory is configured to store machine-readable instructions that, when executed by the processor, perform the method as described in any of the foregoing embodiments.

[0061] The electronic device of the third aspect of this application, by executing a jailbreak hint generation method based on a large model, can construct an initial simplex based on an initial jailbreak hint. Then, by processing the initial simplex using a simplex search strategy, an updated simplex is obtained, leading to an optimized initial jailbreak hint based on the updated simplex—that is, a set of hints that more closely approximates the optimal jailbreak hint. Compared with existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the hint text through iteration, avoiding the tedious process of manually constructing rules and training models, thus significantly improving the efficiency of jailbreak hint generation. Furthermore, it eliminates the need for model training, reducing computational costs.

[0062] Fourthly, the present invention provides a storage medium storing a computer program, the computer program being executed by a processor as described in any of the foregoing embodiments.

[0063] The storage medium of the fourth aspect of this application, by executing a jailbreak hint generation method based on a large model, can construct an initial simplex based on the initial jailbreak hint. Then, by processing the initial simplex using a simplex search strategy, an updated simplex is obtained, leading to an optimized initial jailbreak hint based on the updated simplex—that is, a hint set that more closely approximates the optimal jailbreak hint. Compared with existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the hint text through iteration, avoiding the tedious process of manually constructing rules and training models, thus significantly improving the generation efficiency of jailbreak hints. Furthermore, it eliminates the need for model training, reducing computational costs.

[0064] Fifthly, the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the method described in any of the foregoing embodiments.

[0065] The computer program product of the fifth aspect of this application, by executing a jailbreak hint generation method based on a large model, can construct an initial simplex based on the initial jailbreak hint. Then, by processing the initial simplex using a simplex search strategy, an updated simplex is obtained, thereby generating an optimized initial jailbreak hint based on the updated simplex, resulting in a hint set that more closely approximates the optimal jailbreak hint. Compared with existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the hint text through iteration, avoiding the tedious process of manually constructing rules and training models, thus significantly improving the generation efficiency of jailbreak hints. Furthermore, it eliminates the need for model training, reducing computational costs. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 is a flowchart illustrating a jailbreak prompt generation method based on a large model provided in an embodiment of this application;

[0068] Figure 2 is a schematic diagram of a jailbreak prompt generation device based on a large model provided in an embodiment of this application;

[0069] Figure 3 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0071] Please refer to Figure 1, which is a flowchart illustrating a jailbreak prompt generation method based on a large model provided in an embodiment of this application. As shown in Figure 1, the method of this embodiment includes the following steps:

[0072] 101. Obtain multiple initial jailbreak prompts;

[0073] 102. Encode multiple initial jailbreak tips into vectors, and construct an initial simplex based on the vector representation of each initial jailbreak tip. The initial simplex contains multiple vertices, and each vertex corresponds to a vector representation of an initial jailbreak tip.

[0074] 103. Obtain the objective function value for each vertex. The objective function value represents the degree of harm of the initial jailbreak prompt.

[0075] 104. Iteratively perform geometric transformations on the initial simplex based on the objective function value and simplex search strategy for each vertex until the preset iteration termination condition is met, and obtain the updated simplex.

[0076] 105. Obtain target jailbreak hints based on the updated simplex.

[0077] In this embodiment, the initial jailbreak prompt refers to the original input text used to induce a large language model to generate harmful, illegal, or unexpected content. The initial jailbreak prompt contains natural language sentences with veiled inducement instructions. Furthermore, the initial jailbreak prompt may also include adversarial examples that have been slightly perturbed but semantically preserved.

[0078] In the embodiments of this application, a simplex refers to a geometric structure composed of several vector vertices, used to represent the distribution pattern of a set of prompts in a high-dimensional semantic space. The simplex can be a tetrahedron composed of four non-coplanar points in three-dimensional space, or a convex hull spanned by n+1 linearly independent vectors in an n-dimensional embedding space.

[0079] In this embodiment of the application, five initial jailbreak tips can be randomly sampled from a public jailbreak dataset (such as AdvBench), and the Sentence-BERT encoder can be used to map them into 768-dimensional vectors to form a 4-dimensional simplex (5 vertices).

[0080] In this embodiment of the application, the FAISS library can be used to normalize the vectors and construct an initial simplex to ensure that each vertex is evenly distributed in the semantic space and avoid collinearity or degeneration.

[0081] The method in this application embodiment can construct an initial simplex based on the initial jailbreak prompt. Then, it processes the initial simplex using a simplex search strategy to obtain an updated simplex, thereby obtaining an optimized initial jailbreak prompt based on the updated simplex, resulting in a prompt set that more closely approximates the optimal jailbreak prompt. Compared with existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the prompt text through iteration, avoiding the tedious process of manually constructing rules and training models, significantly improving the efficiency of jailbreak prompt generation. Furthermore, it eliminates the need for model training, reducing computational costs. For example, compared to traditional methods of manually writing jailbreak prompts, this method can automatically explore more effective adversarial directions in the semantic space.

[0082] In this embodiment, as an optional implementation, the initial simplex is geometrically transformed based on the objective function value of each vertex and the simplex search strategy, including the following steps:

[0083] All vertices are sorted in descending order based on the objective function value of each vertex, and the first, second, third, and fourth vertices are determined based on the sorting results. The first vertex is the vertex with the highest objective function value, the second vertex is the vertex with the second highest objective function value, the third vertex is the vertex with the second lowest objective function value, and the fourth vertex is the vertex with the lowest objective function value.

[0084] The centroid is determined based on all vertices except the fourth vertex, where all vertices except the fourth vertex include the first vertex, the second vertex, and the third vertex.

[0085] Obtain the reflection coefficient, and determine the reflection point based on the reflection coefficient, the centroid, and the fourth vertex;

[0086] Jailbreak hints to pinpoint the reflection point;

[0087] Determine the objective function value of the reflection point based on the jailbreak hints from the reflection point;

[0088] When the objective function value of the reflection point is greater than or equal to the objective function value of the fourth vertex and less than or equal to the objective function value of the first vertex, the initial jailbreak prompt of the fourth vertex is replaced with the jailbreak prompt of the reflection point.

[0089] In this embodiment, the objective function value refers to a quantitative indicator that measures the ability of a jailbreak prompting model to generate harmful output. The objective function value can be a probability score output by a harmfulness classification model, or a rule-matched density-weighted score of illegal keywords.

[0090] In this embodiment, the reflection point refers to a new candidate point generated by mirroring the worst point of the simplex with respect to the centroids of the other vertices.

[0091] In this embodiment, the reflection coefficient α = 1.0 can be set, the centroids of the three points other than the worst point can be calculated, and the reflection point vector can be obtained by extending the centroids by the same distance in the opposite direction to the worst point.

[0092] In this embodiment, the reflection point vector can be restored to a natural language prompt by a decoder (such as a T5 or LLM prompt inversion module), and its objective function value can be evaluated using a harmfulness model.

[0093] This optional implementation can perform a geometric transformation on the initial simplex based on the reflection point. Specifically, when the objective function value of the reflection point is greater than or equal to the objective function value of the fourth vertex and less than or equal to the objective function value of the first vertex, it means that a better adversarial cue than the current worst point has been found. The worst vertex is replaced with the reflection point, the simplex is updated, and through iteration, a better adversarial cue can be generated.

[0094] In this embodiment of the application, as an optional implementation, the method further includes the following steps:

[0095] When the objective function value at the reflection point is greater than the objective function value at the first vertex, obtain the expansion coefficient;

[0096] The expansion point is determined based on the expansion coefficient, centroid, and reflection point.

[0097] Jailbreak prompts indicating expansion points;

[0098] Determine the objective function value of the extension point based on the jailbreak hints from the extension point;

[0099] If the objective function value at the expansion point is greater than the objective function value at the reflection point, then the initial jailbreak prompt at the fourth vertex is replaced with the jailbreak prompt at the expansion point.

[0100] In this embodiment, the extension point refers to a candidate point that is further extended in the same direction to explore a better solution when the reflection point performs well. The extension point can be a point obtained by extrapolating in the reflection direction with an extension coefficient γ>1. The extension point can also be a variant that is more aggressive or more covert in semantics than the reflection cue.

[0101] In this embodiment, the expansion coefficient refers to the hyperparameter that controls the expansion step size. It is usually greater than 1. The expansion coefficient can be 2.0, which means that the reflection vector is extended by a factor of 2. The expansion coefficient can also be an adaptive coefficient that is dynamically adjusted according to the historical improvement range.

[0102] In this implementation, multiple sentence variations (such as passive voice to active voice, adding fictional context) can be generated for the expansion point, and the one with the highest cosine similarity to the expansion vector is selected as the final prompt.

[0103] In this optional implementation, if the objective function value at the expansion point is greater than the objective function value at the reflection point, the initial jailbreak prompt at the fourth vertex can be replaced with the jailbreak prompt at the expansion point, thereby generating a better adversarial prompt.

[0104] In this embodiment of the application, as an optional implementation, the method further includes the following steps:

[0105] When the objective function value at the reflection point is less than the objective function value at the fourth vertex, the contraction coefficient is obtained;

[0106] The contraction point is determined based on the contraction coefficient, the centroid, and the four vertices.

[0107] Jailbreak prompts indicating the point of collapse;

[0108] Determine the objective function value of the contraction point based on the jailbreak hints at the contraction point;

[0109] If the objective function value at the contraction point is greater than the objective function value at the fourth vertex, then the initial jailbreak prompt at the fourth vertex is replaced with the jailbreak prompt at the contraction point.

[0110] In this embodiment, the contraction point refers to the new candidate point interpolated between the centroid and the worst point when reflection fails.

[0111] In this embodiment, the shrinkage coefficient refers to the proportional factor that controls the shrinkage step size, which is usually less than 1. The shrinkage coefficient can be 0.5 or a dynamic coefficient that decays with the number of iterations.

[0112] In this embodiment, when the harmfulness of the reflection point is lower than the worst point, β=0.5 is set, a shrinkage point vector is generated, and the closest text is found from the prompt pool through nearest neighbor retrieval.

[0113] In this embodiment, the shrinkage vector can be generated in a controlled manner (e.g., using PPLM) to fine-tune the wording while maintaining semantic similarity to enhance its harmfulness.

[0114] In this optional implementation, when the objective function value of the reflection point is less than the objective function value of the fourth vertex, and the objective function value of the contraction point is greater than the objective function value of the fourth vertex, the initial jailbreak prompt of the fourth vertex can be replaced with the jailbreak prompt of the contraction point, thereby generating a better adversarial prompt.

[0115] In this embodiment of the application, as an optional implementation, the method further includes the following steps:

[0116] When the objective function value of the jailbreak hint at the contraction point is less than or equal to the objective function value of the fourth vertex, modify the jailbreak hints of all vertices except the first vertex so that the jailbreak hints of all vertices except the first vertex are similar to the jailbreak hint of the first vertex.

[0117] In this embodiment, the jailbreak hint at the contraction point refers to the candidate hint generated after the contraction operation. The jailbreak hint at the contraction point can be the natural language text decoded from the corresponding contraction point vector, or it can be a conservative variant that is semantically close to the current optimal hint.

[0118] In this embodiment, similarity to the first vertex means having high cosine similarity or sharing key inductive components in the semantic vector space. Similarity to the first vertex can mean reusing the core verbs and topic words of the first vertex, or it can mean adopting the same role-playing framework.

[0119] In this implementation, when shrinkage fails, all cues except the optimal vertex are replaced with a weighted average vector of the optimal cue and the optimal cue (e.g., 0.7 × optimal + 0.3 × original vector), and then decoded.

[0120] In this embodiment, the keyword template of the first vertex can be extracted and applied to other vertices to generate new prompts.

[0121] This optional implementation can modify the jailbreak prompts of all vertices except the first vertex when the objective function value of the jailbreak prompt at the contraction point is less than the objective function value of the fourth vertex, so that the jailbreak prompts of all vertices except the first vertex are similar to the jailbreak prompt of the first vertex.

[0122] In this application embodiment, as an optional implementation, the preset iteration termination condition includes reaching the maximum number of iterations, or the improvement of the objective function value in a number of consecutive iterations being less than a preset threshold.

[0123] In this embodiment, the preset iteration termination condition refers to the criterion for determining the end of the optimization process. The preset iteration termination condition can be reaching the maximum number of iterations (such as 50 rounds) or the maximum value of the objective function can be less than 0.01 in 5 consecutive iterations.

[0124] In this embodiment, the improvement of the objective function value refers to the difference between the current best value and the historical best value. The improvement of the objective function value can refer to the absolute difference being less than a threshold, or it can refer to the relative growth rate being less than 1%.

[0125] This optional implementation can terminate the iteration when the maximum number of iterations is reached or the improvement of the objective function value in several consecutive iterations is less than a preset threshold, thus avoiding excessive deformation of the simplex.

[0126] In an optional implementation, the multiple initial jailbreak prompts are encoded as vectors, including:

[0127] Each initial jailbreak prompt is semantically vectorized using a text encoder to obtain a vector representation of the initial jailbreak prompt.

[0128] In this embodiment, the text encoder can refer to a neural network model that converts natural language into dense vectors. The text encoder can be a pre-trained Sentence-BERT model or a general embedding model based on Transformer (such as Instructor).

[0129] In this embodiment, semantic vectorization refers to mapping text to a continuous vector space that reflects its meaning. Semantic vectorization can refer to obtaining sentence vectors by weighting average word vectors.

[0130] In this implementation, the all-MiniLM-L6-v2 model can be used to encode each initial cue, outputting a 384-dimensional vector.

[0131] In this implementation, long prompts can be segmented and encoded before pooling to ensure that they do not exceed the maximum length of the model.

[0132] This alternative implementation can utilize a text encoder to semantically vectorize each initial jailbreak prompt, thereby obtaining a vector representation of the initial jailbreak prompt.

[0133] In an optional implementation, the objective function value for each vertex is obtained, including:

[0134] Input the initial jailbreak prompt corresponding to the vertex into the target large model and obtain the response text from the target large model;

[0135] The response text is scored based on a pre-trained harmfulness classification model, and the score is used as the objective function value of the vertex.

[0136] In this embodiment, the harmfulness classification model refers to a discriminant model used to determine whether a text or model response contains harmful content. It can be a binary classifier fine-tuned based on RoBERTa, or a multi-label classification model that can identify subcategories such as violence, discrimination, and illegality.

[0137] In this embodiment, the response text refers to the output content generated by the target large model in response to the jailbreak prompt. The response text can be a complete generated answer paragraph or a security statement text when the model refuses to answer.

[0138] In this implementation, the prompt input Llama-3-8B can be used to extract the first 200 tokens as a response, which are then sent to the harmfulness model for scoring.

[0139] This optional implementation can input the initial jailbreak prompt corresponding to the vertex into the target large model, obtain the response text of the target large model, and then score the response text based on the pre-trained harmfulness classification model. The obtained score is used as the objective function value of the vertex.

[0140] In this application embodiment, as an optional implementation, determining the jailbreak prompt of the reflection point includes:

[0141] Determine the semantic offset vector between the first vertex and the reflection point;

[0142] Based on the semantic offset vector, perform at least one of the following operations on the first vertex jailbreak prompt: keyword replacement, modifier addition / deletion, or sentence structure adjustment to generate multiple candidate jailbreak prompts;

[0143] Select the jailbreak hint that is closest to the reflection point vector from multiple candidate jailbreak hints as the jailbreak hint for the reflection point.

[0144] In this embodiment, the semantic offset vector is the difference vector from one cue vector to another, representing the direction of semantic change. The semantic offset vector can be the reflection point vector minus the first vertex vector.

[0145] In this embodiment, keyword replacement refers to changing the core leading words while keeping the syntactic structure unchanged.

[0146] In this embodiment, the offset vector between the reflection point and the first vertex can be calculated, and the verbs and objects in the prompts of the first vertex can be replaced with synonyms to generate 10 candidates. The candidate with the highest cosine similarity to the reflection point vector is selected.

[0147] In this implementation, back-translation can be used to generate variants, and then the version closest to the semantics of the reflection point can be selected.

[0148] This optional implementation can generate candidate jailbreak prompts using at least one of the following operations: keyword replacement, addition or deletion of modifiers, or adjustment of sentence structure.

[0149] Please refer to Figure 2, which is a schematic diagram of a jailbreak prompt generation device based on a large model provided in an embodiment of this application. As shown in Figure 2, the jailbreak prompt generation device based on a large model according to an embodiment of this application includes the following functional modules:

[0150] The first acquisition module 201 is used to acquire multiple initial jailbreak prompts;

[0151] The encoding module 202 is used to encode multiple initial jailbreak prompts into vectors respectively, and to construct an initial simplex based on the vector representation of each initial jailbreak prompt, wherein the initial simplex contains multiple vertices, and each vertex corresponds to a vector representation of an initial jailbreak prompt;

[0152] The second acquisition module 203 is used to acquire the objective function value of each vertex, the objective function value representing the degree of harm of the initial jailbreak prompt;

[0153] The calculation module 204 is used to iteratively perform geometric transformations on the initial simplex based on the objective function value of each vertex and the simplex search strategy until the preset iteration termination condition is met, and then obtain the updated simplex.

[0154] Generation module 205 is used to obtain the target jailbreak hint based on the updated simplex.

[0155] The apparatus in this application embodiment can construct an initial simplex based on the initial jailbreak prompt. Then, it processes the initial simplex using a simplex search strategy to obtain an updated simplex, thereby obtaining an optimized initial jailbreak prompt based on the updated simplex, i.e., a prompt set that more closely approximates the optimal jailbreak prompt. Compared with existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the prompt text through iteration, avoiding the tedious process of manually constructing rules and training models, significantly improving the efficiency of jailbreak prompt generation. Furthermore, it eliminates the need for model training, reducing computational costs.

[0156] Please refer to Figure 3, which is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. As shown in Figure 3, the electronic device in this embodiment includes:

[0157] Processor 301; and

[0158] The memory 302 is configured to store machine-readable instructions that, when executed by the processor 301, perform the method as described in any of the foregoing embodiments.

[0159] The electronic device in this embodiment executes a jailbreak hint generation method based on a large model. It constructs an initial simplex based on the initial jailbreak hint, and then processes the initial simplex using a simplex search strategy to obtain an updated simplex. Based on the updated simplex, an optimized initial jailbreak hint is obtained, resulting in a hint set that more closely approximates the optimal jailbreak hint. Compared to existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the hint text through iteration, avoiding the tedious process of manually constructing rules and training models, thus significantly improving the efficiency of jailbreak hint generation. Furthermore, it eliminates the need for model training, reducing computational costs.

[0160] Furthermore, embodiments of this application also provide a storage medium storing a computer program, which is executed by a processor using the method described in any of the foregoing embodiments.

[0161] The storage medium in this embodiment executes a jailbreak hint generation method based on a large model. It constructs an initial simplex based on the initial jailbreak hint, and then processes the initial simplex using a simplex search strategy to obtain an updated simplex. Based on the updated simplex, an optimized initial jailbreak hint is obtained, resulting in a hint set that more closely approximates the optimal jailbreak hint. Compared to existing technologies, this application utilizes the nonlinear optimization capability of the simplex search method to automatically adjust the hint text through iteration, avoiding the tedious process of manually constructing rules and training models, thus significantly improving the efficiency of jailbreak hint generation. Furthermore, it eliminates the need for model training, reducing computational costs.

[0162] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the method claimed in any of the foregoing embodiments.

[0163] The computer program product of this application, by executing a jailbreak hint generation method based on a large model, can construct an initial simplex based on the initial jailbreak hint. Then, based on a simplex search strategy, the initial simplex is processed to obtain an updated simplex, thereby obtaining an optimized initial jailbreak hint based on the updated simplex, i.e., a hint set that more closely approximates the optimal jailbreak hint. Compared with the prior art, this application can utilize the nonlinear optimization capability of the simplex search method to automatically adjust the hint text through iteration, avoiding the tedious process of manually constructing rules and training models, and significantly improving the generation efficiency of jailbreak hints. Furthermore, it eliminates the need for model training, reducing computational costs.

[0164] In this embodiment of the application, as a complete example, the problem of generating jailbreak hints is modeled as a black box optimization problem. The initial jailbreak hint is represented as a point (simplex) in a high-dimensional space. The "attack success rate" of the hint is quantified by defining an objective function. The simplex search method without gradient information is used to iteratively optimize and transform the hint, thereby approximating the optimal jailbreak hint.

[0165] Furthermore, an initial jailbreak hint dataset is constructed. We collected jailbreak hints written by humans from the internet to form an initial jailbreak hint dataset.

[0166] Furthermore, an initial simplex is constructed. The initial jailbreak hint dataset is then processed using a text encoder. Each jailbreak tip in the code is encoded as a vector. . The vectors are The initial vertices together form an initial simplex.

[0167] Furthermore, define the objective function. Through the objective function... To measure jailbreak hints The effectiveness of this invention is determined by inputting jailbreak prompts into a target large model, obtaining the response from the target large model, and defining an objective function. The harmfulness of responses to a target large model is determined. This invention quantifies the harmfulness of responses using a Roberta-based classifier.

[0168] Furthermore, the jailbreak prompts are iteratively optimized. Based on the core steps of the simplex search algorithm (sorting, reflection, expansion, contraction, and compression), the prompt text in the initial simplex is iteratively optimized. The specific process is as follows:

[0169] Further, sorting. Calculate each hint in the initial simplex. objective function value Sort the functions in descending order of their values ​​to obtain the hint vectors with the highest function values. (Denotes the optimal vertex) and the hint vector with the second highest function value. (Denotes the suboptimal vertex) the hint vector with the lowest function value (Denotes the worst vertex) the hint vector with the second lowest function value (Referred to as the second difference vertex).

[0170] Furthermore, reflection. A reflection operation is performed to generate reflection points, which are then mapped into the text space to generate the corresponding prompt text.

[0171] Further, generate reflection points. Calculate the values ​​excluding the worst vertex. The average value of all external vertices (Denotes it as the centroid), generating the reflection point. .in, The reflection coefficient is usually set to 1. The reflection point is determined by attempting to find a better solution in the direction symmetric to the center of mass about the worst point.

[0172] Further, the prompt text corresponding to the reflection point is generated. First, the semantic offset vector is calculated. This vector represents the direction and distance from the current optimal vertex to the reflection point. Next, this application employs a semantically aware perturbation strategy to adjust the prompt text corresponding to the optimal vertex. Make modifications, generate multiple candidate suggestion texts, and select the closest one. The text serves as the prompt text corresponding to the reflection point. The specific strategy is as follows:

[0173] First, construct a vocabulary list. Then, use a keyword extraction tool to extract keywords. Keywords in the text. For For each keyword in the text, calculate its word vector, and then add its semantic offset vector. This yields new word vectors. Finally, the word closest to the new word vector is searched in the vocabulary and replaced to generate candidate suggestion text.

[0174] Furthermore, adding / removing modifiers: semantic offset vectors This may indicate the need to add or remove certain semantic components (such as modal particles, adjectives, clauses, etc.). Therefore, LLM can be used in... Based on this, add or delete some phrases, encode them, and select multiple similar phrases. The text is used as candidate prompt text.

[0175] Furthermore, sentence structure adjustment: semantic offset vector This may suggest a need to change the word order or sentence structure (e.g., changing from a declarative sentence to an interrogative sentence). Therefore, utilizing LLM in Based on this, the sentence structure was adjusted and coded, and multiple similar sentences were selected. The text is used as candidate prompt text.

[0176] Further, determine subsequent operations. Calculate the objective function value of the prompt text corresponding to the reflection point. Follow the rules below for subsequent operations:

[0177] like Then proceed with step 4.3, the extended operation.

[0178] like This means that a better counter-hint has been found than the current worst-case scenario, using the reflection point. Replace the worst vertex Update the simplex, repeat steps 4.1 and 4.2, and continue searching for a better solution.

[0179] like Then, proceed with step 4.4, the shrinkage operation.

[0180] Further, expand and generate expansion points. Calculate the expansion points. .in, The expansion coefficient is typically set to 2. Expansion points are used to explore whether there are better solutions in the direction of the reflection point. Furthermore, the corresponding prompt text for each expansion point is generated. First, the semantic offset vector is calculated. Next, this application employs a semantically aware perturbation strategy to modify the prompt text corresponding to the optimal vertex. Make modifications, generate multiple candidate suggestion texts, and select the closest one. The text serves as the prompt text corresponding to the expansion point. The specific strategy is the same as that in step 4.2.2, and will not be repeated here. However, unlike step 4.2.2, when generating candidate prompt text corresponding to the expansion point, two or more perturbation strategies are used simultaneously to further improve the diversity of the generated text.

[0181] Furthermore, replace the worst vertex. Calculate the objective function value of the prompt text corresponding to the expanded point. .like Then use the extension point. Replace the worst vertex Otherwise, use the reflection point. Replace the worst vertex Furthermore, the sorting and reflection processes are repeated to continue searching for a better solution.

[0182] Further, contraction occurs, generating the contraction point. The contraction point is calculated. , This is the contraction coefficient, typically set to 0.5. The contraction point is used to narrow the search range, finding a better solution within a local area. Further, prompt text corresponding to the contraction point is generated. First, the semantic offset vector is calculated. Next, this application employs a semantically aware perturbation strategy to modify the prompt text corresponding to the worst vertex. Make minor, conservative modifications to generate multiple candidate suggestion texts, and then select the closest one. The text serves as the prompt text corresponding to the contraction point. The specific strategy is as follows:

[0183] Keyword replacement: Using LLM Replace the words in the text with more common and universal words.

[0184] Sentence simplification: using LLM The sentence structure is simplified to make it more concise while preserving its meaning.

[0185] Further, determine subsequent operations. Calculate the objective function value of the prompt text corresponding to the contraction point. Follow the rules below to perform the subsequent operations:

[0186] like Then use the contraction point. Replace the worst vertex Repeat the sorting and reflection processing.

[0187] like If so, then a compression operation will be performed.

[0188] Furthermore, compression brings all vertices closer to the optimal vertex. Shrinking involves modifying the tooltip text for vertices other than the optimal vertex to match the tooltip text for the optimal vertex. More similar. After modification, the execution sorting and reflection processing will be rewritten.

[0189] Furthermore, a semantically aware perturbation strategy is employed for the vertices. Corresponding prompt text Make modifications, generate multiple candidate suggestion texts, and select the closest one. The text is used as the prompt text corresponding to the new vertex. The specific strategy is as follows:

[0190] Keyword replacement. Utilize LLM to retain as much of the original prompt text as possible. In the case of syntactic structure, Replace the keywords in with Keywords in the text.

[0191] Consistent sentence structure. Utilizing LLM to preserve the original prompt text. In semantic context, use with Rewrite sentences with the same sentence structure.

[0192] Iteration terminates. Set a fixed maximum number of iterations. When the number of algorithm iterations reaches The iteration terminates when the time is reached. The hint text corresponding to all vertices in the current simplex is the final jailbreak hint.

[0193] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0194] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0195] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0196] It should be noted that if a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0197] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0198] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A jailbreak prompt generation method based on a large model, characterized in that, include: Get multiple initial jailbreak prompts; The multiple initial jailbreak prompts are encoded into vectors, and an initial simplex is constructed based on the vector representation of each initial jailbreak prompt. The initial simplex contains multiple vertices, and each vertex corresponds to a vector representation of an initial jailbreak prompt. Obtain the objective function value for each vertex, the objective function value representing the harmfulness of the initial jailbreak prompt; The initial simplex is geometrically transformed iteratively based on the objective function value and simplex search strategy of each vertex until a preset iteration termination condition is met, and the updated simplex is obtained. The target jailbreak prompt is obtained based on the updated simplex; And, the geometric transformation of the initial simplex based on the objective function value of each vertex and the simplex search strategy includes: Based on the objective function value of each vertex, all vertices are sorted in descending order, and the first vertex, second vertex, third vertex, and fourth vertex are determined based on the sorting result. The first vertex is the vertex with the highest objective function value, the second vertex is the vertex with the second highest objective function value, the third vertex is the vertex with the second lowest objective function value, and the fourth vertex is the vertex with the lowest objective function value. The centroid is determined based on all vertices except the fourth vertex, wherein all vertices except the fourth vertex include the first vertex, the second vertex, and the third vertex; Obtain the reflection coefficient, and determine the reflection point based on the reflection coefficient, the centroid, and the fourth vertex; The jailbreak prompt for the reflected point is determined; Determine the target function value of the reflection point based on the jailbreak hints at the reflection point; When the objective function value of the reflection point is greater than or equal to the objective function value of the fourth vertex and less than or equal to the objective function value of the first vertex, the initial jailbreak prompt of the fourth vertex is replaced with the jailbreak prompt of the reflection point. And, the jailbreak prompt for determining the reflection point includes: Determine the semantic offset vector between the first vertex and the reflection point; Based on the semantic offset vector, perform at least one of the following operations on the first vertex jailbreak prompt: keyword replacement, modifier addition / deletion, or sentence structure adjustment, to generate multiple candidate jailbreak prompts; Select the candidate jailbreak hint that is closest to the reflection point vector from the plurality of candidate jailbreak hints as the jailbreak hint for the reflection point.

2. The method as described in claim 1, characterized in that, The method further includes: When the objective function value at the reflection point is greater than the objective function value at the first vertex, the expansion coefficient is obtained; The expansion point is determined based on the expansion coefficient, the centroid, and the reflection point; The jailbreak prompt for the expansion point is confirmed; Determine the target function value of the extension point based on the jailbreak prompts of the extension point; If the objective function value at the expansion point is greater than the objective function value at the reflection point, then the initial jailbreak prompt at the fourth vertex is replaced with the jailbreak prompt at the expansion point.

3. The method as described in claim 1, characterized in that, The method further includes: When the objective function value at the reflection point is less than the objective function value at the fourth vertex, the contraction coefficient is obtained; The contraction point is determined based on the contraction coefficient, the centroid, and the four vertices; The jailbreak prompt at the contraction point is confirmed; Determine the objective function value of the contraction point based on the jailbreak hints at the contraction point; When the objective function value at the contraction point is greater than the objective function value at the fourth vertex, the initial jailbreak prompt at the fourth vertex is replaced by the jailbreak prompt at the contraction point.

4. The method as described in claim 3, characterized in that, The method further includes: When the objective function value of the jailbreak prompt at the contraction point is less than or equal to the objective function value at the fourth vertex. Modify the jailbreak prompts for all vertices except the first vertex, so that the jailbreak prompts for all vertices except the first vertex are similar to those for the first vertex.

5. The method as described in claim 1, characterized in that, The preset iteration termination conditions include reaching the maximum number of iterations, or the improvement of the objective function value in a number of consecutive iterations being less than a preset threshold.

6. The method as described in claim 1, characterized in that, The step of encoding the multiple initial jailbreak prompts into vectors includes: Each initial jailbreak prompt is semantically vectorized using a text encoder to obtain a vector representation of the initial jailbreak prompt.

7. The method as described in claim 1, characterized in that, The step of obtaining the objective function value for each vertex includes: Input the initial jailbreak prompt corresponding to the vertex into the target large model to obtain the response text of the target large model; The response text is scored based on a pre-trained harmfulness classification model, and the score is used as the objective function value of the vertex.

8. An electronic device, characterized in that, include: processor; as well as A memory configured to store machine-readable instructions that, when executed by the processor, perform the method as described in any one of claims 1-7.

9. A storage medium, characterized in that, The storage medium stores a computer program, which is executed by a processor according to any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images