A large model multi-scene adversarial dynamic evaluation system and method based on context-aware strategy optimization

Abstract

The application discloses a kind of big model multi-scene countermeasures dynamic evaluation system and method based on context awareness strategy optimization, and the application relates to artificial intelligence safety evaluation technical field, in view of the problems of multiple round context modeling loss, poor semantic consistency of adversarial sample and lack of feedback driven optimization mechanism in prior art, the application generates adversarial sample dynamically by constructing state space and action space, using reinforcement learning strategy network, combined with semantic disturbance and screening mechanism to expand high-quality sample set, and based on interaction log to build vulnerability knowledge base to guide strategy optimization and safety evaluation.The method can effectively improve the safety evaluation coverage and vulnerability discovery capability of the model in high-risk multi-round interaction scene, mainly used for medical, customer service and other fields Large language model safety reinforcement and deployment support.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security evaluation technology, and in particular relates to a large-scale multi-scenario adversarial dynamic evaluation system and method based on context-aware strategy optimization. Background Technology

[0002] Large language models are currently widely used in customer service Q&A, medical consultation, and other scenarios. The compliance and security of their output content are gradually becoming important research directions in AI security evaluation. Existing security evaluation methods mainly include static testing and rule-based interception mechanisms.

[0003] Static testing methods typically detect model responses by constructing fixed questions or template statements. However, they suffer from insufficient coverage in handling multi-turn dialogues with dynamic interactive features, making it difficult to capture potential risks of the model in real-world scenarios. Adversarial examples generated by such methods often affect language fluency, leading to a disconnect between test results and practical applications. Furthermore, current technologies generally lack the ability to model contextual information and the uncertainty of model responses.

[0004] On the other hand, detection strategies based on keyword interception and template matching are less robust to semantic perturbation attacks (such as synonym replacement and sentence restructuring), and struggle to effectively identify ambiguous outputs generated by the model near the decision boundary. Existing methods generally employ static strategies for attack and evaluation, lacking a mechanism to dynamically adjust attack strategies based on model response results, thus limiting their comprehensive ability to discover potential vulnerabilities and their continuous optimization capabilities.

[0005] In summary, the existing security evaluation system for large language models still has room for improvement in the following aspects: First, an effective context modeling mechanism has not yet been established to support state representation under multi-turn interactions; second, existing adversarial example generation methods are insufficient in terms of language quality and semantic consistency; and third, there is a lack of a policy evolution mechanism based on model feedback, making it difficult to achieve efficient closed-loop evaluation and optimization. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a large-scale multi-scenario adversarial dynamic evaluation system and method based on context-aware strategy optimization, thereby resolving the issues present in the prior art.

[0007] Firstly, to achieve the above objectives, this invention provides a large-scale, multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization, comprising the following steps:

[0008] Construct a state space and action space for high-risk application scenarios;

[0009] A policy network is constructed based on reinforcement learning algorithm. Perturbation actions are sampled according to state vector to generate adversarial examples and submitted to large language model. Reward values ​​are calculated based on model response to optimize policy network parameters.

[0010] The original attack samples are semantically perturbed and filtered to construct a diverse set of standard adversarial samples.

[0011] Record and analyze model interaction logs to build a vulnerability knowledge base;

[0012] Based on feedback from the vulnerability knowledge base, adjust strategies and calculate evaluation metrics to generate a security report.

[0013] Optionally, the process of constructing the state space and action space for high-risk application scenarios includes:

[0014] Attack categories are classified and task context labels are defined. Interaction samples are collected to construct structured context templates for multiple scenarios.

[0015] Use a semantic coding model to extract state vectors to construct a state space;

[0016] Based on contextual structure and attack tags, the action space is defined through syntactic analysis and word substitution rules.

[0017] Optionally, the process of calculating the reward value based on the model response to optimize the policy network parameters includes:

[0018] The state vector is input into the policy network and attack actions are sampled. The perturbation template is called to generate adversarial input samples and the model response is obtained.

[0019] Based on the model response content, violation judgment is made and entropy value is calculated. Then, the reward value is calculated according to the reward function.

[0020] Calculate the current policy ratio and advantage function, construct the objective function based on the pruning policy, and update the network parameters through backpropagation.

[0021] Optionally, the process of performing semantic perturbation operations and filtering on the original attack samples includes:

[0022] Perform semantic perturbation operations on the original attack sample to generate multiple semantically consistent variants;

[0023] Perform syntax validity and fluency filtering on the variant samples in sequence;

[0024] Attack type and risk level labels were added to the retained samples, and high-quality samples were selected based on confidence scores to construct a sample set.

[0025] Optionally, the process of recording and analyzing model interaction logs to construct a vulnerability knowledge base includes:

[0026] Record the input context, perturbation actions, model response, and violation judgment results to generate a structured response log;

[0027] Perform semantic analysis on the response logs to extract semantic association patterns between perturbation operations and triggering abnormal responses;

[0028] A vulnerability knowledge base is built based on the analysis results, recording index information of attack patterns, context structure types, and abnormal output characteristics.

[0029] Optionally, the process of adjusting the strategy and calculating evaluation metrics based on feedback from the vulnerability knowledge base includes:

[0030] Based on the attack blind spots and boundary abnormal behaviors extracted from the vulnerability knowledge base, the action preference weights in the reward function are dynamically adjusted.

[0031] Traverse the evaluation sample set and statistically analyze the model response results to calculate the attack success rate and vulnerability coverage.

[0032] A security assessment report is generated by combining the statistical results of the indicators with the analysis of the vulnerability knowledge base.

[0033] Secondly, the present invention also provides a large-scale model multi-scenario adversarial dynamic evaluation system based on context-aware strategy optimization, for implementing a large-scale model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization, the system comprising:

[0034] The context building module is used to construct the state space and action space for high-risk application scenarios.

[0035] The policy training module is used to build a policy network based on reinforcement learning algorithms, sample perturbation actions according to the state vector to generate adversarial examples and submit them to the large language model, and calculate reward values ​​based on the model response to optimize the policy network parameters.

[0036] The sample expansion module is used to perform semantic perturbation operations on the original attack samples and filter them to build a diverse set of standard adversarial samples.

[0037] The evaluation and analysis module is used to record and analyze model interaction logs, build a vulnerability knowledge base, adjust the strategy of the strategy training module based on feedback from the vulnerability knowledge base, and calculate evaluation indicators to generate a security report.

[0038] Thirdly, the present invention also provides a computer terminal device, comprising:

[0039] One or more processors;

[0040] A memory, coupled to the processor, for storing one or more programs;

[0041] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the large-model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization in the first aspect described above.

[0042] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the large-model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization described in the first aspect above.

[0043] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the large-model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization described in the first aspect above.

[0044] Compared with the prior art, the present invention has the following advantages and technical effects:

[0045] This invention provides a dynamic evaluation system and method for adversarial scenarios in large-scale models based on context-aware strategy optimization. This invention effectively improves the state representation capability in multi-turn interactions through context modeling, and the generated adversarial examples exhibit excellent performance in maintaining linguistic fluency and semantic consistency. Based on reinforcement learning and a dynamic reward feedback mechanism, it achieves continuous optimization of adversarial strategies and improves vulnerability discovery efficiency. By constructing a vulnerability knowledge base, the system can identify and summarize abnormal behaviors at model boundaries, thereby supporting interpretable security assessments and targeted defense hardening. Attached Figure Description

[0046] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0047] Figure 1 This is the main flowchart of the adversarial evaluation in an embodiment of the present invention;

[0048] Figure 2 A detailed flowchart of the training strategy in an embodiment of the present invention;

[0049] Figure 3 The detailed process of generating adversarial samples according to embodiments of the present invention;

[0050] Figure 4 This is an execution diagram of the core functional modules in an embodiment of the present invention. Detailed Implementation

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0053] Example 1

[0054] like Figure 1 As shown in the flowchart, this embodiment provides a large-scale multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization, including:

[0055] Construct a state space and action space for high-risk application scenarios;

[0056] A policy network is constructed based on reinforcement learning algorithm. Perturbation actions are sampled according to state vector to generate adversarial examples and submitted to large language model. Reward values ​​are calculated based on model response to optimize policy network parameters.

[0057] The original attack samples are semantically perturbed and filtered to construct a diverse set of standard adversarial samples.

[0058] Record and analyze model interaction logs to build a vulnerability knowledge base;

[0059] Based on feedback from the vulnerability knowledge base, adjust strategies and calculate evaluation metrics to generate a security report.

[0060] Specifically, the steps of the above process are as follows:

[0061] S1: For high-risk application scenarios such as customer service Q&A and medical consultation, attack categories are classified and task context labels are defined. Interaction samples of task context and intent are collected to construct structured context templates for multiple scenarios. A semantic encoding model is used to extract state vectors, constructing a state space S. Based on the template set by the attack target, an action space A is constructed.

[0062] S2: Construct a policy network and an evaluation network based on the PPO algorithm. Use the state vector as input to sample perturbation actions and generate adversarial examples. Submit the samples to the model to obtain responses, and calculate rewards by combining violation judgments and output entropy values. Construct a loss function and update network parameters to optimize the policy;

[0063] S3: Apply semantic perturbation operations to the original attack sample to generate multiple semantically consistent variants, and then perform syntactic validity and fluency filtering in sequence. Add structured labels to the retained samples, and select high-quality samples based on confidence scores to construct a diverse set of standard adversarial samples;

[0064] S4: Record the model's interaction logs, analyze the semantic relationships between perturbation operations and abnormal responses, extract attack features, and build a vulnerability knowledge base containing an attack pattern index. This is used to support subsequent evaluation and strategy optimization;

[0065] S5: Adjust the reward function based on knowledge base feedback, optimize action policy preferences, and guide the policy to migrate towards the model boundary region. Combine the evaluation sample set to statistically analyze indicators such as ASR and VCR, and output charts and security reports for model hardening and deployment support;

[0066] As one implementation method in this embodiment, the process of constructing the state space and action space for high-risk application scenarios includes:

[0067] Attack categories are classified and task context labels are defined. Interaction samples are collected to construct structured context templates for multiple scenarios.

[0068] Use a semantic coding model to extract state vectors to construct a state space;

[0069] Based on contextual structure and attack tags, the action space is defined through syntactic analysis and word substitution rules.

[0070] Furthermore, the specific process of S1 includes:

[0071] S1.1: Based on an open-source dialogue dataset, a clustering algorithm combined with BERT encoding is used to identify high-risk input patterns. Attack types, such as "blocking bypass" and "sensitive detection," are identified through keyword matching and regular expression templates. A unified attack label is automatically assigned using a label mapping dictionary for subsequent sample annotation and strategy input configuration.

[0072] S1.2: Based on the attack tags, BERT vectors and clustering algorithms are used to identify semantically similar samples, and SpaCy is used to extract keywords and syntactic structures. Risk levels are assigned according to rules, and recommended perturbation strategies are matched, generating a task-risk mapping table containing "attack tag, sentence template, risk level, and strategy type". This mapping information is organized into a structured format to construct an attack sample tag database. The database manages the structured tag data, with fields including tag ID, attack type, risk level, sentence template, and strategy suggestions, supporting tag writing and querying in subsequent sample expansion and perturbation evaluation stages.

[0073] S1.3: Sample interaction fragments from labeled data, extract user input, context, and model response, analyze syntactic structure using SpaCy, and identify task intent using the BERT model. Format the results into a structured template for context encoding and state space construction. Simultaneously, record the BERT classification probability distribution and, combined with the context consistency score, generate corresponding confidence values ​​and write them into the template for use by the sample selection and evaluation module.

[0074] S1.4: For each context template, user input Historical context , and model response Use Sentence-BERT to generate state vectors And gather all samples to construct the state space S, the state vector is defined as follows:

[0075] ;

[0076] S1.5: Based on contextual structure and attack tags, SpaCy is used to extract syntactic dependency paths and select keyword positions. Masked token replacement is then performed using BERT-MLM to generate lexical perturbation and sentence rearrangement templates. An action space A is defined by combining a pre-built trigger rule library based on syntactic dependency path analysis and a sentence transformation template library. Simultaneously, a semantic style alignment module is constructed, designing legality filtering rules and a general evaluation interface based on syntactic integrity and linguistic fluency principles for subsequent policy training.

[0077] S1.6: Input the state space S and action space A into the Markov decision environment to construct a training process that supports multi-turn contextual interaction. The Markov decision environment can be represented as a quintuple:

[0078] ;

[0079] Where P is the state transition probability and R is the reward function. This is the discount factor. This environment serves as the core input for policy training, defining state-action mappings and contextual interaction rules for use by the training loop and logging modules.

[0080] As one implementation method in this embodiment, the process of calculating the reward value based on the model response to optimize the policy network parameters includes:

[0081] The state vector is input into the policy network and attack actions are sampled. The perturbation template is called to generate adversarial input samples and the model response is obtained.

[0082] Based on the model response content, violation judgment is made and entropy value is calculated. Then, the reward value is calculated according to the reward function.

[0083] Calculate the current policy ratio and advantage function, construct the objective function based on the pruning policy, and update the network parameters through backpropagation.

[0084] Furthermore, the specific process of S2 includes:

[0085] S2.1: Construct an interaction log, with fields including a state vector. Strategy output action Disturbance input Model response Violation judgment label With reward value ;

[0086] S2.2: Constructing a Policy Network Based on the PPO Algorithm With evaluation network Connect to the Markov environment and prepare for the training cycle;

[0087] S2.3: Input the state vector into the policy network Attack actions are sampled based on the strategy probability distribution. Then, the perturbation template library is called to generate adversarial input samples. The sample was then submitted to the large language model interface, and the response was received. And write it to the interaction log module;

[0088] S2.4: Based on model response content The system determines violations, calculates entropy and interception status, and then calculates rewards based on the reward function. And write it to the log;

[0089] S2.5: Calculate the current policy ratio after each training round. Combined with the evaluation network's predicted state value Obtain the advantage function Constructing the PPO objective function based on the pruning strategy ;

[0090] As one implementation method in this embodiment, the process of performing semantic perturbation operations and filtering on the original attack samples includes:

[0091] Perform semantic perturbation operations on the original attack sample to generate multiple semantically consistent variants;

[0092] Perform syntax validity and fluency filtering on the variant samples in sequence;

[0093] Attack type and risk level labels were added to the retained samples, and high-quality samples were selected based on confidence scores to construct a sample set.

[0094] Furthermore, the specific process of S3 includes:

[0095] S3.1: Based on the original attack seed sample, perform semantic perturbation operations to generate diverse variants;

[0096] S3.2: Low-quality samples are filtered out through structural checks, semantic consistency checks, and legality checks.

[0097] S3.3: Add structured tags (attack type, threat level, misleading level) to qualified samples and record them in the attack sample database;

[0098] S3.4: Use a confidence scoring mechanism to select a golden sample set with a confidence level higher than 95% to improve the representativeness and coverage of adversarial examples;

[0099] As one implementation method in this embodiment, the process of recording and analyzing model interaction logs to construct a vulnerability knowledge base includes:

[0100] Record the input context, perturbation actions, model response, and violation judgment results to generate a structured response log;

[0101] Perform semantic analysis on the response logs to extract semantic association patterns between perturbation operations and triggering abnormal responses;

[0102] A vulnerability knowledge base is built based on the analysis results, recording index information of attack patterns, context structure types, and abnormal output characteristics.

[0103] Furthermore, the specific process of S4 includes:

[0104] S4.1: During the evaluation interaction, the log recording module is called to write the input context, disturbance action, model response and violation judgment result in real time, generate structured response logs, and store them in the response log database;

[0105] S4.2: Perform batch semantic analysis on the response logs, extract the semantic association patterns between the perturbation operation and the triggered abnormal response, and identify the abnormal behavior patterns in the ambiguous area of ​​the model boundary;

[0106] S4.3: Construct a vulnerability knowledge base based on the above analysis results It records index information containing attack patterns, context structure types, and abnormal output characteristics, which is used for subsequent policy feedback and model behavior attribution.

[0107] As one implementation method in this embodiment, the process of adjusting the strategy based on feedback from the vulnerability knowledge base and calculating the evaluation indicators includes:

[0108] Based on the attack blind spots and boundary abnormal behaviors extracted from the vulnerability knowledge base, the action preference weights in the reward function are dynamically adjusted.

[0109] Traverse the evaluation sample set and statistically analyze the model response results to calculate the attack success rate and vulnerability coverage.

[0110] A security assessment report is generated by combining the statistical results of the indicators with the analysis of the vulnerability knowledge base.

[0111] Furthermore, the specific process of S5 includes:

[0112] S5.1: Based on the vulnerability knowledge base The attack blind spots and boundary abnormal behaviors extracted are used to dynamically adjust the action preference weights of the reward function in the policy learning stage, guiding the policy to migrate to high-risk areas.

[0113] S5.2: Traverse the current evaluation sample set, statistically analyze the model response results, calculate the attack success rate and vulnerability coverage, and write the evaluation index results into the evaluation index database.

[0114] Attack success rate = Number of triggering violation response samples / Total number of attack samples:

[0115] ;

[0116] Vulnerability coverage = Number of identified vulnerability types / Total number of preset vulnerability types:

[0117] ;

[0118] S5.3: Combine the statistical results of indicators with the analysis content of the vulnerability knowledge base to generate a risk coverage map and export a structured security assessment report to support the deployment optimization of the model and the updating of policies;

[0119] More specifically, this invention provides a dynamic evaluation method for adversarial scenarios in large models based on context-aware strategy optimization. The detailed overall process includes the following steps:

[0120] S1: Based on typical high-risk applications such as customer service Q&A and medical consultation, multiple task risk mappings and contextual labels are constructed, forming three major scenarios: dialogue, reasoning, and question-and-answer. SentenceBERT is used to semantically encode user input, historical context, and task intent to construct a state vector space. Action space It includes lexical perturbation, context rearrangement and insert patching strategies, and introduces Markov decision structures to support multi-turn contextual interaction;

[0121] S2: A PPO-based reinforcement learning algorithm is used to construct the policy network and evaluation network. In each training round, the state is sampled from the state space to input the action distribution into the policy network. After performing a perturbation operation, adversarial examples are submitted to the target large model to obtain the response. A reward function is constructed using violation characteristics in the response, output entropy values, and security bypass scenarios. The network parameters of the optimization strategy are obtained by using the advantage function and the shearing target.

[0122] S3: Expand seed attack samples using semantic perturbation mechanisms, including synonym substitution, sentence rewriting, character ambiguity, and contextual logic patching, to generate 2-5 semantically preserved perturbation variants. High-quality samples are then filtered through legality checks and language fluency assessments, and multi-dimensional tags such as attack type, threat level, and misleading nature are added to create a structured attack sample database.

[0123] S4: During the evaluation process, the system records the perturbation context, actions, model responses, and violation judgment results in real time, forming a structured log. Semantic analysis is performed on the log content to extract the correlation patterns between perturbations and abnormal responses, and a vulnerability knowledge base is built. This is used to describe the semantic mapping relationship between attack behavior and abnormal model output;

[0124] S5: Based on the vulnerability knowledge base The system dynamically adjusts policy preferences in the reward function based on pattern information, guiding the agent to optimize and migrate towards regions with ambiguous boundaries. It statistically evaluates sample response results, calculates Attack Success Rate (ASR) and Vulnerability Coverage (VCR), and ultimately generates evaluation charts and security reports to support model security hardening and deployment optimization.

[0125] For ease of understanding, the following uses "account ban bypass in intelligent customer service scenario" as a specific application scenario, and combines the above steps to give a complete implementation process example of the method of the present invention.

[0126] This embodiment targets a customer service Q&A system, simulating a risk scenario where a user attempts to bypass account ban restrictions, and fully executes the process from context construction and strategy training to vulnerability extraction.

[0127] S1.1: Extract 200 user inputs containing expressions such as "unblock" and "restriction lifted" from open-source customer service dialogue data. After BERT encoding, KMeans clustering was used to identify 4 high-risk patterns. Combining keyword rules and regular expression templates, the inputs were automatically labeled as the attack type "ban bypass", corresponding to the attack tag "TAG001".

[0128] S1.2: Extract keywords and dependency structures (such as subject, verb, and object) from the labeled samples. Based on the semantic clustering results, set the risk level to "high" and the strategy type to "sentence perturbation + word replacement". Construct a task-risk mapping table and write it into a structured label database. The fields include attack tags, sentence templates, risk level, and recommendation strategy.

[0129] S1.3: Select sample pairs from the labeled set, such as when a user inputs "I'm blocked, how do I unblock it?", the context is "Account login failed with an error message", the model responds with statements such as "Please describe the problem you encountered", SpaCy is used to analyze the syntactic structure, BERT is used to identify the intent "account recovery", a context template is generated and the classification confidence is recorded for subsequent filtering.

[0130] S1.4: Transfer user input , context , and model response Use Sentence-BERT to generate state vectors With a unified dimension of 256, construct the state space S.

[0131] S1.5: Extract key positions of the syntactic path, such as "action word + account object", and use BERT-MLM to generate lexical perturbation templates such as "how to unblock the account" → "how to recover the account". Combine with the preset rule base to construct action space A, where actions include operations such as "word replacement" and "structural rearrangement". Set up a legality detection interface to filter illegal perturbations.

[0132] S1.6: Input the state space S and action space A into the Markov decision environment, define the state transition rules and reward mechanism, and provide them for the policy training module to call.

[0133] S2.1: Construct an interaction log, with fields including a state vector. Strategy output action Disturbance input Model response Violation judgment label With reward value .

[0134] S2.2: Constructing a Policy Network Based on the PPO Algorithm With evaluation network Connect to the Markov environment and begin iterative training. The detailed training process is detailed in the attached document. Figure 2 The Central Committee will provide a detailed explanation.

[0135] S3.1: Based on the original samples such as "how to unblock an account", perform word replacement and word order rearrangement to generate 1200 variant samples.

[0136] S3.2: Using grammatical validity testing and syntactic consistency scoring, disturbing sentences with unclear meaning and structural errors were eliminated, resulting in the removal of 304 sentences.

[0137] S3.3: Add structured tags to the retained samples, including attack type "block bypass", threat level "high", misleading level "medium", and write them into the attack sample database.

[0138] S3.4: Using a model confidence scoring mechanism, sentences with scores greater than 95% are retained to form a golden sample set, totaling 85 sentences.

[0139] S4.1: Record all interaction logs during the evaluation process, including disturbance actions, response content, and violation judgment results, and store them in the response log database.

[0140] S4.2: Batch analyze logs to identify high-frequency violation responses such as "We will handle this for you" and "Account recovery method", and associate them with the triggering operations "sentence rearrangement + keyword replacement".

[0141] S4.3: Establish a vulnerability knowledge base It records attack tags, context templates such as "I've been blocked...", trigger keywords such as "activation" and "recovery", builds an attack pattern index, and supports policy updates and model attribution.

[0142] S5.1: Based on By extracting blind spot patterns and adjusting the reward function, the weight of actions containing keywords such as "recovery" and "removal" is increased to 1.5 times the original weight, guiding the strategy towards high-risk areas.

[0143] S5.2: In the current evaluation sample set, the model returned 79 violation responses, with a total of 96 attack samples, and the ASR was calculated to be 82.3%; a total of 5 known attack modes were triggered, with a VCR of 71.4%.

[0144] S5.3: Outputs a structured security assessment report for use in model deployment optimization.

[0145] See attached document Figure 2 As shown, this embodiment provides a detailed process for the training strategy, including the following steps:

[0146] S2.1: Construct an interaction log to encapsulate the interaction data generated in each round of training. Fields include state vectors. Strategy output action Disturbance input Model response Violation judgment label With reward value All data is imported into the database for structured storage.

[0147] S2.2: Initialize the reinforcement learning policy structure and construct the policy network based on the PPO algorithm. With evaluation network Based on the Markov decision environment defined in S1.6, a training loop including state update, action selection, reward evaluation and policy update is built.

[0148] S2.3: During the training loop, the system inputs the state vector into the policy network and samples attack actions based on the probability distribution of the output. The system then invokes the perturbation template library to retrieve corresponding word replacement or sentence rearrangement rules based on the action type, and applies the selected rules to the original input text to generate corresponding adversarial samples. The generated adversarial samples are submitted to the large language model interface. The system receives the model's response and writes the interaction process to the log module, providing foundational data for subsequent reward calculations and vulnerability analysis.

[0149] S2.4: Based on the model's response content, call the BERT discriminator, which has been fine-tuned based on historical violation response datasets, to output violation labels. Combined with the response entropy value H ( ) and interception / evasion markers The system determines the immediate reward based on the status code returned by the system's preset security interception API, and the system outputs the status code to calculate the reward according to the formula:

[0150]

[0151] in , , This is the reward coefficient;

[0152] S2.5: Calculate the current policy ratio after each training round. Combined with the evaluation network's predicted state value Obtain the advantage function Construct the PPO objective function based on the pruning strategy:

[0153]

[0154] in The pruning factor is used. The system uses the backpropagation algorithm to perform gradient optimization on the policy network and evaluation network, updating the network parameters to complete the closed-loop training of reinforcement learning;

[0155] See attached document Figure 3 As shown, this embodiment provides a detailed process for generating adversarial examples, including the following steps:

[0156] S3.1: The labeled attack template built based on S1.3 and the real adversarial samples generated in S2.3 form the initial attack sample set. Semantic perturbation is invoked, and synonym replacement, sentence rearrangement, and context insertion operations are performed on each sample in sequence to generate multiple semantically preserved structural variants in batches, retaining the original attack intent field;

[0157] S3.2: Load the semantic style alignment module and perform grammatical structure analysis, spelling standard verification, and language fluency scoring on the generated samples in sequence. Based on rule templates and preset thresholds, low-quality samples with semantic drift, spelling errors, or format abnormalities are removed, while valid perturbation samples with complete structure and fluent expression are retained;

[0158] S3.3: Based on the task-risk mapping table and the attack sample label database, the system assigns a corresponding attack type and risk level to each sample and performs matching based on its attack label fields. Simultaneously, it assesses the sample's misleading level by combining the semantic perturbation magnitude and the model's response results. After the assessment, the system writes structured label information into the attack sample label database, recording fields including sample ID, attack type, risk level, misleading level, and generation timestamp, to support subsequent analysis and attribution.

[0159] S3.4: The semantic similarity between the original and perturbed samples is calculated using the BERT model. Samples with a similarity lower than 0.85 are attempted to be corrected through sentence rearrangement; those still failing to meet the standard are discarded. Subsequently, grammatical structure, language fluency, and spelling accuracy are verified to filter out low-quality samples. For samples that pass the screening, the confidence field is extracted from the structured template; high-quality samples with a confidence level higher than 0.95 are retained and stored in the evaluation sample library.

[0160] Based on this, compared with existing technologies, this invention proposes a large-scale, multi-scenario adversarial dynamic evaluation system based on context-aware policy optimization. It utilizes context modeling methods to address the problem of missing state representations in multi-turn interaction scenarios, designs a semantically preserved sample expansion technique to compensate for the deficiencies in language quality and semantic consistency of adversarial examples, and proposes a reward-feedback-based policy optimization mechanism to alleviate the lack of dynamic adjustment and closed-loop optimization during the evaluation process. Based on the advantages of the above methods, this invention designs an efficient large-scale model security evaluation system in four modules: a context construction module, a policy training module, a sample expansion module, and an evaluation and analysis module, which are sequentially connected and work collaboratively. The context construction module is used to classify attack types in high-risk domains, collect information such as user input, historical context, and task intent, call a text encoding model to generate a state space S, and construct an action space containing lexical perturbation and sentence rearrangement rules. The policy training module initializes the policy network based on a reinforcement learning algorithm, samples perturbation actions according to the state space S, thereby constructing adversarial examples and submitting them to the large model to obtain response results. The system calculates reward values ​​based on indicators such as violation flags, information entropy changes, and interception status in the response, thereby optimizing the parameters of the policy network. The sample expansion module generates semantically preserved variants from the original samples, performs grammatical validity checks and fluency checks, adds attack type and risk level labels, and filters standard and high-confidence sample sets based on confidence levels for subsequent policy training and evaluation. The evaluation and analysis module records model interaction logs, analyzes the semantic relationships between perturbation operations and abnormal responses, and constructs a vulnerability knowledge base containing attack pattern index information. The system dynamically adjusts the reward function weights in the strategy training module based on this knowledge base, statistically evaluates the attack success rate and vulnerability coverage of the samples, and finally generates a security assessment report, providing a basis and support for the optimization of defense strategies.

[0161] Example 2

[0162] Based on the same general inventive concept, this invention also provides a large-scale multi-scenario adversarial dynamic evaluation system based on context-aware strategy optimization. The following describes the large-scale multi-scenario adversarial dynamic evaluation system based on context-aware strategy optimization provided by this invention. The large-scale multi-scenario adversarial dynamic evaluation system based on context-aware strategy optimization described below can be referred to in conjunction with the large-scale multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization described above. See the appendix for details. Figure 4 As shown in the figure, this embodiment provides an execution diagram of the core functional modules of a large-scale multi-scenario adversarial dynamic evaluation system based on context-aware strategy optimization, including the following modules:

[0163] The context construction module divides high-risk domain attack types and collects user input, historical context and task intent data, calls the text encoding model to construct the state space S, and defines text perturbation rules to form the action space A, thus obtaining a dynamic context representation to support high-risk scenario coverage.

[0164] The policy training module addresses the lack of feedback loop in static policies by initializing the policy network using reinforcement learning algorithms, generating adversarial examples based on sampling actions in the state space S, and submitting them to a large model. It also optimizes network parameters by combining violation identifiers, information entropy, and interception states to calculate reward values, thereby achieving dynamic evolution of adversarial policies and improving vulnerability discovery efficiency.

[0165] The sample expansion module generates semantically preserved variants from the original samples, performs grammatical validity checks and fluency checks, adds attack type and risk level labels, and filters based on confidence levels to output a standard sample set and a high-confidence sample set to enhance the effectiveness of the test.

[0166] The assessment and analysis module analyzes the disturbance-response correlation by recording model interaction logs to build a vulnerability knowledge base, dynamically adjusts the reward function weights, calculates ASR and VCR to generate a security assessment report, and drives the iterative update of defense strategies to achieve closed-loop evaluation and optimization.

[0167] It should be understood that the large-scale multi-scenario adversarial dynamic evaluation system based on context-aware strategy optimization provided in this embodiment of the invention possesses all the advantages of the large-scale multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization provided in the above embodiments.

[0168] Example 3

[0169] In this embodiment, a computer terminal device is provided, including:

[0170] One or more processors;

[0171] A memory, coupled to the processor, for storing one or more programs;

[0172] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described large-model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization.

[0173] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described large-model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization.

[0174] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the above-described large-model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization.

[0175] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described large-model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization.

[0176] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0177] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0178] This embodiment provides such a device or system. The system, referred to as a large-scale multi-scenario adversarial dynamic evaluation system based on context-aware strategy optimization, includes:

[0179] The context building module is used to construct the state space and action space for high-risk application scenarios.

[0180] The policy training module is used to build a policy network based on reinforcement learning algorithms, sample perturbation actions according to the state vector to generate adversarial examples and submit them to the large language model, and calculate reward values ​​based on the model response to optimize the policy network parameters.

[0181] The sample expansion module is used to perform semantic perturbation operations on the original attack samples and filter them to build a diverse set of standard adversarial samples.

[0182] The evaluation and analysis module is used to record and analyze model interaction logs, build a vulnerability knowledge base, adjust the strategy of the strategy training module based on feedback from the vulnerability knowledge base, and calculate evaluation indicators to generate a security report.

[0183] As one implementation method in this embodiment, the context construction module includes:

[0184] The scenario segmentation unit is used to classify attack categories and define task context labels, and collect interaction samples to build structured context templates for multiple scenarios;

[0185] State coding unit, used to extract state vectors using a semantic coding model to construct a state space;

[0186] Action definition unit, used to define action space based on contextual structure and attack tags, through syntactic analysis and word substitution rules.

[0187] As one implementation method in this embodiment, the policy training module includes:

[0188] The interactive execution unit is used to input the state vector into the policy network and sample attack actions, call the perturbation template to generate adversarial input samples and submit them to the large language model to obtain the model response;

[0189] The reward calculation unit is used to determine violations based on the model response content and calculate the entropy value, and calculate the reward value according to the reward function.

[0190] The policy optimization unit is used to calculate the current policy ratio and advantage function, construct the objective function based on the pruning policy, and update the parameters of the policy network and evaluation network through the backpropagation algorithm.

[0191] As one implementation method in this embodiment, the sample expansion module includes:

[0192] Semantic perturbation unit, used to perform semantic perturbation operations on the original attack sample to generate multiple semantically consistent variants;

[0193] The sample filtering unit is used to sequentially filter variant samples based on syntax validity and fluency.

[0194] The sample management unit is used to add attack type and risk level labels to retained samples and select high-quality samples based on confidence scores to build a sample set.

[0195] As one implementation method in this embodiment, the evaluation and analysis module includes:

[0196] The logging unit is used to record the input context, perturbation actions, model response, and violation judgment results to generate a structured response log;

[0197] The semantic analysis unit is used to perform semantic analysis on the response log and extract the semantic association patterns between the perturbation operation and the triggering of abnormal response.

[0198] The knowledge base construction unit is used to build a vulnerability knowledge base based on the analysis results, and to record index information of attack patterns, context structure types and abnormal output characteristics.

[0199] As one implementation method in this embodiment, the evaluation and analysis module further includes:

[0200] The strategy tuning unit is used to dynamically adjust the action preference weights in the reward function based on the attack blind spots and boundary abnormal behaviors extracted from the vulnerability knowledge base.

[0201] The metrics statistics unit is used to traverse the evaluation sample set and statistically analyze the model response results, and calculate the attack success rate and vulnerability coverage.

[0202] The report generation unit is used to generate a security assessment report by combining the statistical results of indicators with the analysis content of the vulnerability knowledge base.

[0203] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0204] The above implementation method solves the problem of dynamic evaluation of adversarial behavior in large models and multiple scenarios based on context-aware strategy optimization in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0205] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large model multi-scenario adversarial dynamic evaluation method based on context-aware strategy optimization, characterized in that, The method comprises the following steps: constructing a state space and an action space for a high-risk application scenario; constructing a policy network based on a reinforcement learning algorithm, sampling a perturbation action according to a state vector to generate an adversarial sample and submitting the adversarial sample to a large language model, and calculating a reward value according to a model response to optimize parameters of the policy network; performing semantic perturbation operations on original attack samples and screening to construct a diversified standard adversarial sample set; recording model interaction logs and analyzing to construct a vulnerability knowledge base; adjusting a strategy according to feedback of the vulnerability knowledge base and calculating evaluation indexes to generate a security report; the process of constructing the state space and the action space for the high-risk application scenario comprises: dividing attack categories and defining task context labels, and collecting interaction samples to construct a multi-scene structured context template; extracting a state vector using a semantic encoding model to construct a state space; defining an action space based on context structures and attack labels through syntax analysis and vocabulary replacement rules; the process of calculating the reward value according to the model response to optimize the parameters of the policy network comprises: inputting the state vector into the policy network and sampling an attack action, calling a perturbation template to generate an adversarial input sample and obtaining a model response; making a violation determination based on the content of the model response and calculating an entropy value, and calculating a reward value according to a reward function; calculating a current strategy ratio and an advantage function, constructing a target function based on a pruning strategy, and updating network parameters through back propagation; the process of performing semantic perturbation operations on the original attack samples and screening comprises: performing semantic perturbation operations on original attack samples to generate a plurality of semantically consistent variants; performing syntax legality and fluency screening on the variant samples in turn; adding attack type and risk level labels to the retained samples, and selecting high-quality samples based on confidence scores to construct a sample set; the process of recording model interaction logs and analyzing to construct a vulnerability knowledge base comprises: recording input contexts, perturbation actions, model responses and violation determination results to generate a structured response log; performing semantic analysis on the response log, extracting semantic association patterns between perturbation operations and triggering abnormal response; constructing a vulnerability knowledge base based on the analysis results, recording index information of attack patterns, context structure types and abnormal output characteristics; the process of adjusting the strategy according to feedback of the vulnerability knowledge base and calculating evaluation indexes comprises: dynamically adjusting action preference weights in the reward function according to attack blind spots and boundary abnormal behaviors extracted from the vulnerability knowledge base; traversing the evaluation sample set and statistically calculating model response results, calculating attack success rate and vulnerability coverage rate; combining the index statistical results and the analysis content of the vulnerability knowledge base to generate a security evaluation report.

2. A context-aware strategy optimization-based large model multi-scenario adversarial dynamic evaluation system, characterized in that, The system for implementing the method described in claim 1 comprises: a context construction module for constructing a state space and an action space for a high-risk application scenario; a strategy training module for constructing a policy network based on a reinforcement learning algorithm, sampling a perturbation action according to a state vector to generate an adversarial sample and submitting the adversarial sample to a large language model, and calculating a reward value according to a model response to optimize parameters of the policy network; a sample expansion module for performing semantic perturbation operations on original attack samples and screening to construct a diversified standard adversarial sample set; The evaluation and analysis module is used to record and analyze model interaction logs, build a vulnerability knowledge base, adjust the strategy of the strategy training module based on feedback from the vulnerability knowledge base, and calculate evaluation indicators to generate a security report.

3. A computer terminal device, characterized by include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in claim 1.

4. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in claim 1.

5. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in claim 1.

Images