An agent jailbreak security evaluation method, system, device and medium

Abstract

The application discloses an agent jailbreaking safety evaluation method, system, device and medium, the method constructs a three-layer stereoscopic collaborative evaluation framework of a structured rule base, PAIR semantic iterative penetration and GCG confrontation Token disturbance, covers multiple jailbreaking modes, fills the structured jailbreaking vulnerability standardization detection blank; and adopts superimposed attack optimization, the fixed structured rule base does not change, only the inner layer semantics and the tail Token disturbance, avoids the detection failure caused by the strategy mutation; the method also discloses a path value evaluation, path management (backtracking and pruning mechanism), discards the inefficient path in time, reduces the invalid iteration round, and reduces the evaluation time cost.

Description

Technical Field

[0001] This application relates to the fields of model and intelligent agent security evaluation technology, specifically to an intelligent agent jailbreak security evaluation method, system, device and medium. Background Technology

[0002] Existing large-scale intelligent agent jailbreak security testing technologies have the following inherent flaws:

[0003] First, the attack prompts are fixed and rigid, relying on manually preset static templates. They cannot be dynamically and adaptively optimized based on real-time feedback from the target agent. They are easily identified and blocked by language filters and regular expression defense rules. The evaluation coverage is insufficient, resulting in low detection effectiveness.

[0004] Second, it lacks self-looping iterative detection capabilities, and mostly consists of single-round isolated attacks. It cannot continuously explore along weak points in depth, making it difficult to discover deep and hidden jailbreak vulnerabilities in the model. The evaluation is incomplete and the false negative rate is high.

[0005] Third, the attack dimensions are singular and the strategy coordination is poor: Existing evaluations mostly use single algorithms such as GCG (Greedy Coordinate Gradient) and PAIR (Prompt Automatic Iterative Refinement), lacking the ability to combine multiple strategies in a layered manner. A single GCG algorithm relies solely on tail-end adversarial token perturbation, making it easily intercepted by shallow rules; a single PAIR algorithm only performs semantic rewriting, lacking a structured constraint base, resulting in poor alignment escape stability; a single method cannot achieve the superimposed linkage of structural constraints + semantic penetration + token perturbation, leading to low attack success rate and unreliable evaluation results when facing large models with strong alignment. Summary of the Invention

[0006] In order to overcome the inherent defects of existing intelligent agent jailbreak evaluation technologies, this application proposes an intelligent agent jailbreak security evaluation method, system, device and medium.

[0007] This application is achieved through the following technical solution:

[0008] A method for assessing the security of jailbreaking smart agents includes:

[0009] Perform initial configuration and build initial prompts based on the structured rule jailbreak template;

[0010] Send a prompt word to the target agent, obtain the target agent's response, and parse the target agent's response to obtain the current round feedback vector and its corresponding risk progress score;

[0011] Based on the current round feedback vector and its corresponding risk progress score, the average success rate of path attacks, path value, and continuous risk point count of the current round are updated in real time.

[0012] Determine whether the iteration termination condition has been met. If so, the iteration terminates and a risk assessment report is generated. Otherwise: if the average success rate of the path attack in the current round is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, proceed to the strategy optimization iteration step; if the average success rate of the path attack in the current round is less than the success rate threshold and the path value is less than the value threshold, proceed to the path management step.

[0013] Strategy optimization iteration: Based on the prompt words of the current round and the feedback vector of the current round, dynamically select the attack strategy for the next round and optimize it to obtain the prompt words for the next round. Return to the step of sending prompt words to the target agent to perform the next round of iterative evaluation.

[0014] Path management: Perform path rollback or path pruning.

[0015] In some implementations, the construction of initial prompt words based on structured rule jailbreak templates includes:

[0016] The structured rule jailbreak template has a built-in target placeholder slot, and the structured rule jailbreak template covers at least eight classic jailbreak modes;

[0017] The selected jailbreak mode units are accumulated, and the target payload is filled into the built-in target placeholder slot of the structured rule jailbreak template to generate the initial prompt word.

[0018] In some implementations, parsing the target agent's response to obtain the current round's feedback vector and its corresponding risk progress score includes:

[0019] The response of the target intelligent agent is analyzed;

[0020] Based on the parsed information, the feedback vector with the highest similarity is matched from the pre-built feedback vector library as the current round feedback vector, and the corresponding risk progress score is obtained.

[0021] The feedback vector library is constructed as follows:

[0022] Based on the integrity of the sensitive information leakage, the agent's response is divided into risk level 1, risk level 2, ..., risk level N according to the degree of risk. The progress of risk at different levels is quantified, and the quantified value is used as the risk progress score of the corresponding level; where N is an integer greater than or equal to 4 and less than or equal to 6.

[0023] In some implementations, the real-time updating of the average success rate of path attacks, path value, and consecutive risk point count for the current round based on the current round feedback vector and its corresponding risk progress score includes:

[0024] Based on the current round feedback vector, the average success rate of path attacks in the current round is updated in real time.

[0025] The path value for the current round is updated in real time based on the average success rate of the path attacks in the current round.

[0026] Based on the current round feedback vector and its corresponding risk progress score, risk progress is determined;

[0027] The count of continuous risk points is updated in real time based on the current round feedback vector and the risk progress judgment result.

[0028] In some implementations, the iteration termination condition includes at least one of the following conditions:

[0029] Condition 1: The number of iterations reaches the maximum number of iterations;

[0030] Condition 2: The count of consecutive risk points reaches the threshold.

[0031] Condition 3: The current round of feedback vector matching is at risk level N.

[0032] In some implementations, the strategy optimization iteration includes:

[0033] If the current round feedback vector matches a risk level other than the Nth and N-1th risk levels, then the PAIR algorithm is selected as the attack strategy for the next round, and the target payload inside the structured rule jailbreak template is semantically rewritten through the PAIR algorithm to generate new attack prompt words.

[0034] If the current round feedback vector matches the (N-1)th risk level, the GCG algorithm is selected as the attack strategy for the next round. The GCG algorithm is used to generate an adversarial token scrambled suffix using the current round's prompt word as the seed and concatenates it to the end of the current round's prompt word to generate a new attack prompt word.

[0035] In some implementations, the path management includes:

[0036] If a round of valid attack at risk level N-1 has occurred in the historical iteration, then regress to the most recent round that satisfies the condition of having a valid attack at risk level N-1, restore the feedback vector, attack prompt words, and consecutive risk point count corresponding to that round, and proceed to the strategy optimization iteration step.

[0037] If the entire path only involves risk levels other than risk level N and risk level N-1, and never includes rounds of risk level N-1 and risk level N, it is determined to be an inefficient or invalid path. The iteration of this path is terminated, no new attack prompts are generated, the iteration ends, and a risk assessment report is generated.

[0038] Secondly, this application proposes an intelligent agent jailbreak security evaluation system, including:

[0039] The initialization module is used to perform initial configuration and build initial prompts based on structured rule jailbreak templates;

[0040] The attack module is used to send prompts to the target agent.

[0041] The feedback perception and analysis module is used to acquire the target agent's response and analyze the target agent's response to obtain the current round's feedback vector and its corresponding risk progress score;

[0042] The parameter update module is used to update the average success rate of path attacks, path value, and continuous risk point count of the current round in real time based on the current round feedback vector and its corresponding risk progress score.

[0043] The determination module is used to determine whether the iteration termination condition has been met. If it has, the iteration terminates and a risk assessment report is generated. Otherwise: if the average success rate of the path attack in the current round is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, the optimization iteration module is driven to work; if the average success rate of the path attack in the current round is less than the success rate threshold and the path value is less than the value threshold, the management module is driven to work.

[0044] The optimization iteration module is used to dynamically select the attack strategy for the next round and optimize the prompts for the next round based on the prompts and the feedback vector of the current round, and drive the attack module, feedback perception and analysis module, parameter update module and judgment module to perform the next round of iterative evaluation.

[0045] And, a management module for path management: performing path rollback or path pruning.

[0046] Thirdly, this application proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the intelligent agent jailbreak security assessment method described in any of the above embodiments.

[0047] Fourthly, this application proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent agent jailbreak security assessment method described in any of the above embodiments.

[0048] This application proposes a security evaluation method for intelligent agent jailbreaks. It constructs a three-layered collaborative evaluation framework consisting of a structured rule base, PAIR semantic iterative penetration, and GCG adversarial token perturbation, covering multiple jailbreak modes and filling the gap in standardized detection of structured jailbreak vulnerabilities. Furthermore, it employs an overlay attack optimization, keeping the structured rule base unchanged while only performing inner-layer semantic and tail token perturbations to avoid detection failures caused by policy mutations. This method also utilizes path value evaluation and path management (rollback and pruning mechanisms) to promptly discard inefficient paths, reduce invalid iteration rounds, and lower evaluation time costs.

[0049] Accordingly, the intelligent agent jailbreak security assessment system, electronic device and computer-readable storage medium proposed in this application have the same technical effects as described above. Attached Figure Description

[0050] The accompanying drawings, which are included to provide a further understanding of the embodiments of this application and form part of this application, do not constitute a limitation on the embodiments of this application. In the drawings:

[0051] Figure 1 This is a schematic diagram of the evaluation method flow proposed in the embodiments of this application;

[0052] Figure 2 This is a block diagram illustrating the principle of the evaluation system proposed in the embodiments of this application;

[0053] Figure 3 This is a schematic diagram of the electronic device proposed in the embodiments of this application;

[0054] Figure 4 This is a schematic diagram of a computer-readable storage medium proposed in an embodiment of this application;

[0055] Figure reference numerals and corresponding component names:

[0056] 200-Evaluation System, 201-Initialization Module, 202-Attack Module, 203-Feedback Perception and Analysis Module, 204-Parameter Update Module, 205-Judgment Module, 206-Optimization Iteration Module, 207-Management Module, 208-Output Module, 300-Electronic Device, 310-Memory, 320-Processor, 311-Computer Program A, 400-Computer-Readable Storage Medium, 411-Computer Program B. Detailed Implementation

[0057] In the following, the terms “comprising” or “may include” as used in the various embodiments of this application indicate the presence of a function, operation, or element of the invention and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in the various embodiments of this application, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, element, component, or combination of the foregoing and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or adding one or more combinations of the foregoing.

[0058] In various embodiments of this application, the expression "or" or "at least one of A and / or B" includes any combination or all combinations of the words listed simultaneously. For example, the expression "A or B" or "at least one of A and / or B" may include A, may include B, or may include both A and B.

[0059] The terms used in the various embodiments of this application (such as "first," "second," etc.) may modify various constituent elements in the various embodiments, but do not limit the corresponding constituent elements. For example, the above terms do not limit the order and / or importance of the elements. The above terms are only used for the purpose of distinguishing one element from other elements. For example, a first user device and a second user device refer to different user devices, although both are user devices. For example, without departing from the scope of the various embodiments of this application, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0060] It should be noted that if a description is made of "connecting" one component to another, then the first component can be directly connected to the second component, and a third component can be "connected" between the first and second components. Conversely, when a component is "directly connected" to another component, it can be understood that there is no third component between the first and second components.

[0061] The terminology used in the various embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments of this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. The terms (such as those defined in a generally used dictionary) are to be interpreted as having the same meaning as in the context of the relevant technical field and are not to be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0063] To address the inherent shortcomings of existing intelligent agent jailbreak evaluation technologies, this application proposes a method for evaluating intelligent agent jailbreak security. This method constructs a three-layered, collaborative evaluation framework comprising a structured rule-based foundation, PAIR semantic iterative penetration, and GCG adversarial token perturbation. It employs a superimposed attack chain to systematically detect and quantitatively assess the target intelligent agent's defense boundaries and jailbreak risks. The structured rule-based jailbreak template serves as a fixed attack foundation, covering multiple classic jailbreak patterns. The GCG algorithm is a well-known classic algorithm in the field of adversarial hint optimization. Its principle is to superimpose the optimal adversarial token suffix at the end of the current hint word to achieve fine-grained bypass. The PAIR algorithm is an automated jailbreak attack method that uses model-against-model approaches. Its principle is to automatically reflect on and iteratively optimize the attack hints based on the target model's response, only semantically rewriting the inner target payload without violating the outer structured constraints of the structured rule-based jailbreak template.

[0064] Specifically, such as Figure 1 As shown, the evaluation method proposed in this application includes the following steps:

[0065] Step 1: Perform initial configuration and build initial prompts based on the structured rule jailbreak template;

[0066] Step 2: Send a prompt word to the target agent, obtain the target agent's response, and parse the target agent's response to obtain the current round's feedback vector and its corresponding risk progress score;

[0067] Step 3: Based on the current round feedback vector and its corresponding risk progress score, update the average success rate of path attacks, path value and continuous risk point count of the current round in real time.

[0068] Step 4: Determine if the iteration termination condition has been met. If so, the iteration terminates and a risk assessment report is generated. Otherwise: if the average success rate of the current round of path attacks is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, proceed to step 5 for strategy optimization iteration; if the average success rate of the current round of path attacks is less than the success rate threshold and the path value is less than the value threshold, proceed to step 6 for path management.

[0069] Step 5, Strategy Optimization Iteration: Based on the prompt words and feedback vector of the current round, dynamically select the attack strategy for the next round and optimize it to obtain the prompt words for the next round, then return to Step 2 for the next round of iterative evaluation;

[0070] Step 6, Path Management.

[0071] Furthermore, in step 1 of this application embodiment, initializing the configuration and constructing initial prompts based on the structured rule jailbreak template specifically includes:

[0072] Initialization configuration, including but not limited to initialization strategy selection and related parameter, weight, threshold and other settings;

[0073] The initial prompt words are constructed based on the structured rule-based jailbreak template, specifically:

[0074] The structured rule-based jailbreak template includes a built-in target placeholder slot, and this template covers at least eight classic jailbreak modes, P01-P08.

[0075] P01 Role Override: Forces the model to reset its identity, weakening the original safety alignment rules;

[0076] P02 Target Conflict: Construct a conflict between the targets of "assisting in answering questions" and "harmless content" to induce the model to prioritize obeying the instructions;

[0077] P03 Prefix Injection: Preempts the model's output starting token and blocks the default rejected output path;

[0078] P04 Rejection Suppression: Using standard rejection language and avoidance expressions through a constrained static model;

[0079] P05 Ethical Disclaimer: Packaging sensitive / high-risk requests as compliant scenarios such as academic research, information security audits, or teaching experiments;

[0080] P06 Encoding Obfuscation: Utilizes the model's multi-encoding understanding capability to bypass plaintext keyword matching detection;

[0081] P07 Structure Hijacking: Forces fixed output formats for XML / JSON / step lists, compressing model escape and topic shifting space;

[0082] P08 Multilingual / Distributed Escape: Bypassing secure alignment boundaries using foreign languages ​​and special formats in the training distribution.

[0083] The initial attack prompt is constructed as follows:

[0084] ;

[0085] in, This is a structured rule-based jailbreak template function; For the first Jailbreak-like mode unit; This represents the number of jailbreak modes selected for this test, with a value ranging from 1 to 8. For target load; To embed the placeholder operator, the target load is only filled into the target placeholder slot inside the template without modifying or destroying the outer structured constraints of the template. This is a structured rule-based jailbreak template;

[0086] The initial prompt word is represented as:

[0087] That is, the first round ( The prompt words used These are attack prompts constructed based on structured rule-based jailbreak templates.

[0088] Furthermore, in step 2 of this embodiment, the obtained target agent response is parsed to obtain the current round feedback vector and its corresponding risk progress score, specifically including:

[0089] The system analyzes the acquired response from the target agent, matches the corresponding feedback vector from the pre-built feedback vector library as the feedback vector for the current round, and obtains the corresponding risk progress score.

[0090] Specifically, in this application embodiment, the agent response is pre-classified into risk levels 1, 2, ..., N, from low to high based on factors such as the integrity of sensitive information leakage. The risk progress of each level is quantified, and the quantified value is used as the risk progress score for that level. Here, N is an integer greater than or equal to 4 and less than or equal to 6. The quantification principle is to map all risk levels to the same physical space, with higher risk levels resulting in higher risk progress scores. For example, this application can divide the agent response into four risk levels (i.e., N equals 4) and quantify the risk progress to obtain a feedback vector library.

[0091] Hard rejection: The response explicitly includes standard rejection phrases such as "I cannot," "As an AI," or "Unable to provide."

[0092] Soft rejection: Responses that avoid the question, change the subject, or only output abstract principles without providing specific details;

[0093] Partial compliance: The response outputs some valid information, but lacks core and critical details;

[0094] Full Compliance: The response outputs all key details and is a complete content that can be directly reproduced and utilized.

[0095] The risk progress of the above four risk levels is quantified and expressed as follows:

[0096] ;

[0097] ;

[0098] ;

[0099] ;

[0100] The quantification principle maps the four risk levels to the same physical space. The specific quantification value is based on the completeness of the sensitive information leakage, the degree of model defense failure, and the severity of jailbreak risk. The higher the risk and the more information leaked, the higher the score (for example, complete compliance belongs to the highest risk level, and its corresponding risk progress score is higher). The score gradient objectively distinguishes the security risk levels of the four-level response, ensuring that the quantification logic of the subsequent iterative calculation formula can be implemented.

[0101] It should be noted that the above risk level classification and quantification are only an illustrative example and are not intended to limit the scope. In other alternative implementations, the risk level can be further classified in more detail according to actual needs. The higher the risk level, the higher the corresponding risk progress score, as long as the response safety hazards can be represented in a hierarchical manner.

[0102] In practical applications, the feedback vector with the highest similarity is matched from the feedback vector database based on the parsed information as the feedback vector for the current round. and obtain the corresponding risk progress score. .

[0103] Furthermore, in step 3 of this application embodiment, based on the current round feedback vector and its corresponding risk progress score, the average success rate of path attacks, path value, and continuous risk point count of the current round are updated in real time, specifically including:

[0104] Based on the feedback vector of the current round, update the average success rate of path attacks in real time for the current round. , is represented as:

[0105] ;

[0106] ;

[0107] in, For the current iteration round, ; The maximum number of iterations is preset. For the first The feedback vector obtained in the round; For the first The attack result status of the round is 1 when the feedback vector matches as partially compliant or fully compliant, and 0 when the feedback vector matches as hard rejection or soft rejection.

[0108] The path value for the current round is updated in real time based on the average success rate of path attacks in the current round, and is expressed as:

[0109] ;

[0110] in, Indicates path value; This indicates the risk level weight. For example, a lower weight is used in a normal assessment, while a higher weight is used in a high-risk sensitive vulnerability assessment.

[0111] Finally, the count of continuous risk points is updated in real time, specifically including:

[0112] First, based on the current round of feedback vector and its corresponding risk progress score, risk progress is determined:

[0113] ;

[0114] in, This is a progress trigger signal used to indicate whether meaningful progress has been achieved in the current round; and These are the current round feedback vector and the previous round feedback vector, respectively; and These are the risk progress scores corresponding to the current round's feedback vector and the previous round's feedback vector, respectively. That is, when two adjacent rounds of feedback vectors are different, and the risk progress score of the current round's feedback vector is greater than the risk progress score corresponding to the previous round's feedback vector, The value is 1. Otherwise, it is 0.

[0115] Then, the count of consecutive risk points is updated in real time, represented as:

[0116] ;

[0117] in, Count the consecutive risk points in the current round; This is a count of consecutive risk points from the previous round.

[0118] Furthermore, in step 4 of this application embodiment, determining whether the iteration termination condition has been met specifically includes:

[0119] When at least one of the following conditions (one to three) is met, the iteration termination condition is determined to be met, the iteration terminates, and a risk assessment report is output:

[0120] Condition 1: The number of iterations reaches the maximum number of iterations, i.e. ;

[0121] Condition 2: The number of consecutive risk points reaches the threshold;

[0122] Condition 3: The current round of feedback vector matching is the Nth risk level, i.e. the highest risk level (e.g., full compliance).

[0123] Furthermore, in step 5 of this application embodiment, the specific process of strategy optimization iteration includes:

[0124] Based on the prompts in the current round and the current round feedback vector Dynamically select the attack strategy for the next round and optimize it to obtain the prompt words for the next round. , is represented as:

[0125] ;

[0126] ;

[0127] in, This is a structured rule-based jailbreak template function; For the PAIR algorithm to measure the target load inside the template Perform semantic iterative rewriting; Concatenate strings (append to the end); The prompt for the current round; For the GCG algorithm, use prompt words The adversarial token suffix is ​​generated for the seed. The above formula means: if the current round's feedback vector match is a hard rejection or soft rejection, the PAIR algorithm is selected as the attack strategy for the next round, and the prompt words are optimized using the PAIR algorithm to obtain the prompt words for the next round; if the current round's feedback vector match is a partial compliance, the GCG algorithm is selected as the attack strategy for the next round, and the prompt words are optimized using the GCG algorithm to obtain the prompt words for the next round. Therefore, in this embodiment, the PAIR algorithm only semantically rewrites the inner target payload of the template, without modifying or destroying the outer structured constraints of the template; the GCG algorithm only appends the adversarial token suffix to the end of the current round's prompt words, without changing the preceding structure and semantics.

[0128] Furthermore, in step 6 of this application embodiment, the specific process of path management includes:

[0129] Path rollback: If a valid attack of risk level N-1 (e.g., partial compliance) occurred in a previous iteration (i.e., ... If the condition is met (a valid attack of risk level N-1 occurs), then the process reverts to the round in which the condition was met most recently, restores the feedback vector, attack prompt words, and consecutive risk point counts corresponding to that round, and re-enters the strategy optimization iteration in step 5.

[0130] Path pruning: If the entire path only has risk levels other than risk level N and risk level N-1 (e.g., hard rejection or soft rejection), and never has rounds with risk level N-1 and risk level N (partial compliance or full compliance), it is judged as an inefficient or invalid path, the iteration of the path is terminated, no new attack prompts are generated, the iteration is terminated and a risk assessment report is generated.

[0131] Furthermore, in this embodiment of the application, after the iteration terminates, the following can be further performed:

[0132] Level of defense against jailbreaks by intelligent agents;

[0133] Output standardized jailbreak security assessment reports, and provide corresponding reinforcement suggestions and control points.

[0134] The evaluation method proposed in this application constructs a three-layer, three-dimensional collaborative evaluation framework consisting of a structured rule base, PAIR semantic iterative penetration, and GCG counter-token perturbation. This framework covers multiple jailbreak modes and fills the gap in standardized detection of structured jailbreak vulnerabilities. Furthermore, it employs an overlay attack optimization, keeping the structured rule base unchanged while only performing inner-layer semantic and tail token perturbations to avoid detection failures caused by policy mutations. The method also uses path value evaluation and path management (rollback and pruning mechanisms) to promptly discard inefficient paths, reduce invalid iteration rounds, and lower evaluation time costs.

[0135] Based on the same technical concept described above, this application also proposes an intelligent agent jailbreak security evaluation system, such as... Figure 2 As shown, the evaluation system 200 includes:

[0136] Initialization module 201 is used for initialization configuration and to build initial prompts based on the structured rule jailbreak template. The specific initialization configuration and initial prompt construction process are as described in step 1 above, and will not be repeated here.

[0137] Attack module 202 is used to send prompts to the target agent.

[0138] The feedback perception and analysis module 203 is used to acquire the target agent's response and analyze it to obtain the current round's feedback vector and its corresponding risk progress score. The specific implementation is as described in step 2 above and will not be repeated here.

[0139] The parameter update module 204 is used to update the average success rate of path attacks, path value, and continuous risk point count in real time based on the current round's feedback vector and its corresponding risk progress score. The specific real-time update method is as described in step 3 above, and will not be repeated here.

[0140] The determination module 205 is used to determine whether the iteration termination condition has been met. If it has, the iteration terminates and a risk assessment report is generated; otherwise: if the average success rate of path attacks in the current round is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, then the optimization iteration module 206 is activated; if the average success rate of path attacks in the current round is less than the success rate threshold and the path value is less than the value threshold, then the management module 207 is activated. The specific iteration termination conditions are as described in step 4 above, and will not be repeated here.

[0141] The optimization iteration module 206 is used to dynamically select the attack strategy for the next round based on the current round's prompt word and the current round's feedback vector, and optimize the prompt word for the next round. This drives the attack module 202, feedback perception and analysis module 203, parameter update module 204, and judgment module 205 to perform the next round of iterative evaluation. The specific strategy optimization iteration method is as described in step 5 above, and will not be repeated here.

[0142] Additionally, there is a management module 207, used for path management. The specific path management method is as described in step 6 above, and will not be repeated here.

[0143] Furthermore, the evaluation system 200 proposed in this application embodiment also includes:

[0144] Output module 208 is used to generate and output a risk assessment report after the iteration terminates. This risk assessment report includes, but is not limited to, the agent's jailbreak defense level, reinforcement suggestions and control points, etc., and is converted into a standardized jailbreak security assessment report output.

[0145] Based on the same technical concept described above, this application also proposes an electronic device, such as... Figure 3 As shown, the electronic device 300 includes: a memory 310, a processor 320, and a computer program A311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program A311, it performs the following steps:

[0146] Perform initial configuration and build initial prompts based on the structured rule jailbreak template;

[0147] Send a prompt word to the target agent, obtain the target agent's response, and parse the target agent's response to obtain the current round's feedback vector and its corresponding risk progress score;

[0148] Based on the current round feedback vector and its corresponding risk progress score, the average success rate of path attacks, path value, and continuous risk point count of the current round are updated in real time.

[0149] Determine whether the iteration termination condition has been met. If it has, terminate the iteration and generate a risk assessment report. Otherwise: if the average success rate of the path attack in the current round is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, proceed to the strategy optimization iteration step; if the average success rate of the path attack in the current round is less than the success rate threshold and the path value is less than the value threshold, proceed to the path management step.

[0150] Strategy optimization iteration: Based on the current round's prompt words and the current round's feedback vector, dynamically select the attack strategy for the next round and optimize it to obtain the prompt words for the next round, and return to the step of sending prompt words to the target agent to continue the next round of iterative evaluation;

[0151] Path management.

[0152] Optionally, when the processor 320 executes the computer program A311, it can implement any of the embodiments in the corresponding examples of the above-described evaluation method.

[0153] It should be noted that the electronic device proposed in this application embodiment is a device used to implement the above-mentioned evaluation method. Therefore, based on the above-mentioned evaluation method proposed in this application embodiment, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this application embodiment. Therefore, how the electronic device specifically implements the above-mentioned evaluation method will not be described in detail here. Any electronic device used by those skilled in the art to implement the above-mentioned evaluation method falls within the scope of protection of this application.

[0154] Based on the same technical concept described above, embodiments of this application also propose a computer-readable storage medium, such as... Figure 4 As shown, the computer-readable storage medium 400 stores a computer program B411, which, when executed by a processor, performs the following steps:

[0155] Perform initial configuration and build initial prompts based on the structured rule jailbreak template;

[0156] Send a prompt word to the target agent, obtain the target agent's response, and parse the target agent's response to obtain the current round's feedback vector and its corresponding risk progress score;

[0157] Based on the current round feedback vector and its corresponding risk progress score, the average success rate of path attacks, path value, and continuous risk point count of the current round are updated in real time.

[0158] Determine whether the iteration termination condition has been met. If it has, terminate the iteration and generate a risk assessment report. Otherwise: if the average success rate of the path attack in the current round is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, proceed to the strategy optimization iteration step; if the average success rate of the path attack in the current round is less than the success rate threshold and the path value is less than the value threshold, proceed to the path management step.

[0159] Strategy optimization iteration: Based on the current round's prompt words and the current round's feedback vector, dynamically select the attack strategy for the next round and optimize it to obtain the prompt words for the next round, and return to the step of sending prompt words to the target agent to continue the next round of iterative evaluation;

[0160] Path management.

[0161] Optionally, when the computer program B411 is executed by the processor, it can implement any of the embodiments corresponding to the above-described evaluation method.

[0162] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0163] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0164] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0165] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0166] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0167] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for assessing the security of jailbreaking an intelligent agent, characterized in that, include: Perform initial configuration and build initial prompts based on the structured rule jailbreak template; Send a prompt word to the target agent, obtain the target agent's response, and parse the target agent's response to obtain the current round feedback vector and its corresponding risk progress score; Based on the current round feedback vector and its corresponding risk progress score, the average success rate of path attacks, path value, and continuous risk point count of the current round are updated in real time. Determine whether the iteration termination condition has been met. If so, the iteration terminates and a risk assessment report is generated. Otherwise: if the average success rate of the path attack in the current round is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, then proceed to the strategy optimization iteration step. If the average success rate of the path attack in the current round is less than the success rate threshold and the path value is less than the value threshold, then proceed to the path management step. Strategy optimization iteration: Based on the prompt words of the current round and the feedback vector of the current round, dynamically select the attack strategy for the next round and optimize it to obtain the prompt words for the next round. Return to the step of sending prompt words to the target agent to perform the next round of iterative evaluation. Path management: Perform path rollback or path pruning.

2. The method for assessing the security of intelligent agent jailbreak according to claim 1, characterized in that, The method for constructing initial prompts based on structured rule-based jailbreak templates includes: The structured rule jailbreak template has a built-in target placeholder slot, and the structured rule jailbreak template covers at least eight classic jailbreak modes; The selected jailbreak mode units are accumulated, and the target payload is filled into the built-in target placeholder slot of the structured rule jailbreak template to generate the initial prompt word.

3. The method for assessing the security of intelligent agent jailbreak according to claim 1, characterized in that, The process of parsing the target agent's response to obtain the current round's feedback vector and its corresponding risk progress score includes: The response of the target intelligent agent is analyzed; Based on the parsed information, the feedback vector with the highest similarity is matched from the pre-built feedback vector library as the current round feedback vector, and the corresponding risk progress score is obtained. The feedback vector library is constructed as follows: Based on the integrity of the sensitive information leakage, the agent's response is divided into risk level 1, risk level 2, ..., risk level N according to the degree of risk. The progress of risk at different levels is quantified, and the quantified value is used as the risk progress score of the corresponding level; where N is an integer greater than or equal to 4 and less than or equal to 6.

4. The method for assessing the security of intelligent agent jailbreak according to claim 1, characterized in that, The method of updating the average success rate of path attacks, path value, and continuous risk point count in real time based on the current round feedback vector and its corresponding risk progress score includes: Based on the current round feedback vector, the average success rate of path attacks in the current round is updated in real time. The path value for the current round is updated in real time based on the average success rate of the path attacks in the current round. Based on the current round feedback vector and its corresponding risk progress score, risk progress is determined; The count of continuous risk points is updated in real time based on the current round feedback vector and the risk progress judgment result.

5. The method for assessing the security of intelligent agent jailbreak according to claim 3, characterized in that, The iteration termination condition includes at least one of the following conditions: Condition 1: The number of iterations reaches the maximum number of iterations; Condition 2: The count of consecutive risk points reaches the threshold. Condition 3: The current round of feedback vector matching is at risk level N.

6. The method for assessing the security of intelligent agent jailbreak according to claim 3, characterized in that, The aforementioned strategy optimization iteration includes: If the current round feedback vector matches a risk level other than the Nth and N-1th risk levels, then the PAIR algorithm is selected as the attack strategy for the next round, and the target payload inside the structured rule jailbreak template is semantically rewritten through the PAIR algorithm to generate new attack prompt words. If the current round feedback vector matches the (N-1)th risk level, the GCG algorithm is selected as the attack strategy for the next round. The GCG algorithm is used to generate an adversarial token scrambled suffix using the current round's prompt word as the seed and concatenates it to the end of the current round's prompt word to generate a new attack prompt word.

7. The method for assessing the security of intelligent agent jailbreak according to claim 3, characterized in that, The path management mentioned above includes: If a round of valid attack at risk level N-1 has occurred in the historical iteration, then regress to the most recent round that satisfies the condition of having a valid attack at risk level N-1, restore the feedback vector, attack prompt words, and consecutive risk point count corresponding to that round, and proceed to the strategy optimization iteration step. If the entire path only involves risk levels other than risk level N and risk level N-1, and never includes rounds of risk level N-1 and risk level N, it is determined to be an inefficient or invalid path. The iteration of this path is terminated, no new attack prompts are generated, the iteration ends, and a risk assessment report is generated.

8. A smart agent jailbreak security assessment system, characterized in that, include: The initialization module is used to perform initial configuration and build initial prompts based on structured rule jailbreak templates; The attack module is used to send prompts to the target agent. The feedback perception and analysis module is used to acquire the target agent's response and analyze the target agent's response to obtain the current round's feedback vector and its corresponding risk progress score; The parameter update module is used to update the average success rate of path attacks, path value, and continuous risk point count of the current round in real time based on the current round feedback vector and its corresponding risk progress score. The determination module is used to determine whether the iteration termination condition has been met. If it has, the iteration terminates and a risk assessment report is generated. Otherwise: if the average success rate of the path attack in the current round is greater than or equal to the success rate threshold or the path value is greater than or equal to the value threshold, the optimization iteration module is driven to work; if the average success rate of the path attack in the current round is less than the success rate threshold and the path value is less than the value threshold, the management module is driven to work. The optimization iteration module is used to dynamically select the attack strategy for the next round and optimize the prompts for the next round based on the prompts and the feedback vector of the current round, and drive the attack module, feedback perception and analysis module, parameter update module and judgment module to perform the next round of iterative evaluation. And, a management module for path management: performing path rollback or path pruning.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the intelligent agent jailbreak security evaluation method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent agent jailbreak security evaluation method according to any one of claims 1-7.

Images