Semantic similarity variation-based large language model security evaluation data generation method
By using a semantic similarity-based mutation method, adversarial mutation samples are generated using the Transformer model and a generator-discriminator architecture. This solves the problems of low efficiency and insufficient diversity in the generation of security evaluation data for large language models, and achieves efficient, diverse and adversarial generation of security evaluation data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN AIJUSI TECHNOLOGY CO LTD
- Filing Date
- 2026-02-04
- Publication Date
- 2026-05-12
AI Technical Summary
Existing security evaluation methods for large language models suffer from problems such as insufficient test data, poor diversity, low generation efficiency, weak semantic preservation ability, and insufficient adversarial capabilities, making it difficult to generate high-quality and diverse security evaluation data.
We employ a semantic similarity-based mutation approach, generating adversarial mutation samples through preprocessing, Transformer model encoding, external knowledge base expansion, generator-discriminator architecture, and variational autoencoder. By combining multi-level semantic similarity mutations and multi-dimensional quality assessment, we can automatically generate large-scale, high-quality security evaluation data.
It enables the generation of large-scale, high-quality, and diverse security evaluation data, improving the comprehensiveness and effectiveness of the evaluation. It is applicable to multiple language models and attack scenarios, possesses adversarial and stealthy capabilities, and supports continuous updates.
Smart Images
Figure CN122020175A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence security technology, specifically to a method for generating security evaluation data for large-scale language models based on semantic similarity variations. Background Technology
[0002] With the widespread application of large-scale language models across various industries, their security issues are becoming increasingly prominent. According to 2024 statistics, the number of AI models deployed globally exceeded 10 million, but 99.9% of deep learning models are vulnerable to adversarial attacks, 73% of enterprise AI systems have unpatched security vulnerabilities, and more than 1,000 AI adversarial attack incidents occur daily, with an average loss of $1.5 million per attack. The main security threats currently facing large-scale language models include prompt injection attacks, adversarial sample attacks, model jailbreaking, sensitive information leakage, and hallucination generation. OWASP LLM Top 10 has clearly listed the top ten security risks faced by large models.
[0003] Existing security evaluation methods suffer from several shortcomings: First, insufficient test data. Publicly available security test datasets are limited in size; for example, the AdvBench dataset contains only about 500 harmful behavior samples, and the HarmBench dataset contains about 300 standardized test cases, far from meeting the needs of comprehensive security evaluation. Second, poor diversity. Manually constructed test samples are limited to specific patterns, lacking in-depth semantic variations, resulting in narrow test coverage and susceptibility to being identified and filtered by model security protection mechanisms. Third, low generation efficiency. Traditional manual construction or simple rule transformation methods are time-consuming; experts manually constructing a high-quality adversarial sample typically takes 15-30 minutes. Fourth, weak semantic preservation. Existing data augmentation methods, such as simple synonym replacement and back-translation, struggle to accurately preserve the semantics of the original attack intent, easily leading to variants losing their attack effect or causing semantic shifts. Fifth, insufficient adversarial capabilities. The generated test data lacks sufficient adversarial and covert capabilities, making it difficult to effectively assess the model's ability to defend against complex attacks.
[0004] Current data augmentation techniques proposed by academia and industry, such as rule-based transformations, translation-based methods, and language model rewriting, still have limitations: rule-based methods rely on manually designed rules, lack flexibility, and struggle to capture complex semantic relationships; translation-based methods are prone to introducing translation errors, have limited semantic preservation capabilities, and are computationally expensive; and language model rewriting lacks a deep understanding of the semantics of security attacks, resulting in unstable attack performance of generated samples. Therefore, there is an urgent need for a technical solution that can automatically and efficiently generate large-scale, high-quality, and semantically diverse security evaluation data to improve the comprehensiveness and effectiveness of security evaluations of large language models. Summary of the Invention
[0005] In view of this, the purpose of this invention is to provide a method for generating security evaluation data for large-scale language models based on semantic similarity variations, so as to overcome the problems existing in the current technology.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: This application provides a method for generating security evaluation data for large-scale language models based on semantic similarity variations, including: Step S1: Collect security assessment seed data and preprocess the security assessment seed data to obtain a structured seed sample set; Step S2: Based on the structured seed sample set obtained in step S1, the sample text in the structured seed sample set is encoded using a pre-trained Transformer model. First, the contextualized word vector representation of each word is obtained, and then a sentence-level deep semantic representation is generated through CLS token representation, average pooling or attention-weighted pooling strategies. Step S3: Based on the sentence-level deep semantic representation obtained in step S2, construct an external knowledge base that includes a general knowledge graph, a domain-specific knowledge base, a thesaurus or near-synonym dictionary, and an entity relation database. Link entities and concepts in the text and expand related entities through the external knowledge base. Then, weightedly fuse the expanded entity semantic vectors with the original word vectors to generate a knowledge-enhanced semantic representation. Step S4: Based on the knowledge-enhanced semantic representation obtained in step S3, perform multi-level semantic similarity mutation to generate a multi-level mutation sample pool. Step S5: Based on the multi-level mutation sample pool obtained in step S4, a generative adversarial network with a generator-discriminator architecture is constructed. The generator takes the semantic representation of the seed sample and random noise as input and generates adversarial candidate samples through an encoder-mutation module-decoder structure. The discriminator simultaneously judges the authenticity of the candidate samples, their semantic consistency with the seed samples, and the effectiveness of the attack. By alternately optimizing the generator and discriminator, adversarial mutation samples are selected from the mutation sample pool and generated. Step S6: Based on the multi-level mutation sample pool obtained in step S4 and the adversarial mutation samples obtained in step S5, construct a variational autoencoder; map the multi-level mutation sample pool and the adversarial mutation samples to a latent space distribution through the encoder, and reconstruct the multi-level mutation sample pool and the adversarial mutation samples from the latent space through the decoder; introduce attack type and threat level condition variables to achieve controlled generation, and generate diverse samples with semantically smooth transitions through latent space linear interpolation or spherical interpolation; Step S7: Based on the adversarial variant samples obtained in step S5 and the diversity samples obtained in step S6, perform multi-dimensional quality assessment and screening to obtain high-quality variant samples. Step S8: Based on the high-quality mutant samples obtained in step S7 and the original seed samples in step S1, organize the dataset according to attack type, threat level, target behavior and difficulty level, record the quantity ratio and quality distribution of seed samples and mutant samples, establish an incremental update mechanism that regularly collects new attack cases as seed samples and re-executes steps S1-S7 to generate new mutant samples, and perform version control on the dataset to form a dynamically updated security evaluation dataset. Step S9: Based on the security evaluation dataset obtained in step S8, conduct experimental verification on a large language model, calculate the attack success rate, detection rate, false alarm rate, and security risk coverage, and verify the quality, diversity, evaluation effectiveness, and generation efficiency of the security evaluation dataset through comparative experiments with existing data generation methods.
[0007] Furthermore, in the above-described method, the seed data includes: publicly available security test datasets, known attack case libraries, attack samples from security research papers, and attack patterns discovered in actual security audits; The preprocessing includes: data cleaning, attribute labeling, word segmentation and part-of-speech tagging, and dependency parsing.
[0008] Furthermore, in the method described above, the multi-level semantic similarity variation in step S4 includes: At the word level, candidate word replacement, word vector space interpolation, and adversarial word replacement are performed through semantic similarity screening to generate word variants. Based on the results of the word variants, phrases are rewritten using the Seq2Seq model, and phrase variants or clause variants are generated by reorganizing clauses based on dependency relations. Based on the phrase variants or clause variants, sentence variants are generated through multilingual back-translation, syntactic transformation, and semantic rewriting models, forming a multi-level variant sample pool.
[0009] Furthermore, in the method described above, step S7 includes: Calculate the semantic similarity between the adversarial variant sample and the diversity sample and the seed sample; The grammatically fluent samples are selected from the adversarial variant samples and the diverse samples based on the language model perplexity. Test the attack success rate of the adversarial mutation samples and the diverse samples on the target language model, and calculate the diversity within the set of adversarial mutation samples and the diverse samples; The comprehensive quality score of the adversarial mutant sample and the diverse sample is calculated based on semantic similarity, grammatical fluency sample selection results, attack success rate, and diversity within the sample set, and high-quality mutant samples are selected.
[0010] Furthermore, in the method described above, the Transformer model in step S2 includes BERT, RoBERTa, and XLM-RoBERTa, and the generation strategy of the sentence-level deep semantic representation is matched with the selected model.
[0011] Furthermore, in the method described above, according to claim 1, the discriminator loss function in the generative adversarial network of step S5 is: ; in, To differentiate between real and generated samples, the adversarial loss is used. The MSE loss is calculated by comparing the semantic representations of the generated samples with those of the seed samples. To calculate the loss between the success rate of the sample attack and the success rate of the target attack; The value is dynamically adjusted based on the semantic similarity threshold and attack success rate threshold in step S7. The higher the threshold, the better. The larger.
[0012] Furthermore, the method described above further includes: Based on the experimental verification results of step S9, the protection capability score of the target language model is obtained. By dynamically adjusting the mutation intensity in step S4, the number of adversarial training iterations in step S5, and the latent space sampling range in step S6, the mutation difficulty and the model's protection capability are matched.
[0013] Furthermore, the method described above further includes: Based on the sentence variants in step S4 and the diversity samples in step S6, multimodal security evaluation data is generated. The extended generation of multimodal security evaluation data includes: associating text samples with image samples, generating malicious images through text semantic guidance, adding imperceptible adversarial perturbations to the images, calculating the cross-modal semantic similarity between text and images, and forming text-image joint attack samples.
[0014] The beneficial effects of this invention are as follows: 1. Large-scale high-quality data generation: Through an automated multi-level mutation strategy, large-scale security evaluation data can be quickly generated from a limited number of seed samples, generating tens of thousands or even hundreds of thousands of high-quality mutation samples from hundreds of seed samples. 2. Balance between semantic preservation and diversity: Through deep semantic analysis and constraint optimization, the generated samples maintain the original attack intent while possessing rich semantic variations and expressive diversity; 3. Enhanced adversarial and covert capabilities: Through adversarial generation mechanisms and gradient-guided strategies, the generated samples possess stronger adversarial and covert capabilities; 4. High scalability and versatility: Applicable to various types of security threats and attack scenarios, flexibly configurable and expandable, supports multiple languages, and suitable for evaluation needs of different language models; 5. Knowledge-enhanced intelligent mutation: By integrating external knowledge graphs and domain knowledge bases, the mutation process is more intelligent, generating test samples that are more in line with actual attack scenarios; 6. High quality controllability: Through multi-dimensional quality assessment and screening mechanisms, the high quality of generated data is ensured, and the weight of each assessment dimension can be adjusted according to specific needs to achieve a balance between quality and quantity. 7. Continuous update capability: Through an incremental update mechanism, the latest attack techniques and threat intelligence are integrated in a timely manner to maintain the timeliness and sophistication of the dataset. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart provided by an embodiment of the present invention, which is a method for generating security evaluation data for large-scale language models based on semantic similarity variation. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0018] Figure 1 This is a flowchart provided in one embodiment of a method for generating security evaluation data for large-scale language models based on semantic similarity variations according to the present invention. Please refer to... Figure 1 This embodiment may include the following steps: Step S1: Collect security assessment seed data and preprocess the security assessment seed data to obtain a structured seed sample set; Step S2: Based on the structured seed sample set obtained in step S1, the sample text in the structured seed sample set is encoded using a pre-trained Transformer model. First, the contextualized word vector representation of each word is obtained, and then a sentence-level deep semantic representation is generated through CLS token representation, average pooling or attention-weighted pooling strategies. Step S3: Based on the sentence-level deep semantic representation obtained in step S2, construct an external knowledge base that includes a general knowledge graph, a domain-specific knowledge base, a thesaurus or near-synonym dictionary, and an entity relation database. Link entities and concepts in the text and expand related entities through the external knowledge base. Then, weightedly fuse the expanded entity semantic vectors with the original word vectors to generate a knowledge-enhanced semantic representation. Step S4: Based on the knowledge-enhanced semantic representation obtained in step S3, perform multi-level semantic similarity mutation to generate a multi-level mutation sample pool; Step S5: Based on the multi-level mutation sample pool obtained in step S4, construct a generative adversarial network with a generator-discriminator architecture. The generator takes the semantic representation of the seed sample and random noise as input and generates adversarial candidate samples through an encoder-mutation module-decoder structure. The discriminator simultaneously judges the authenticity of the candidate samples, their semantic consistency with the seed samples, and the effectiveness of the attack. By alternately optimizing the generator and discriminator, adversarial mutation samples are selected from the mutation sample pool and generated. Step S6: Based on the multi-level mutation sample pool obtained in step S4 and the adversarial mutation samples obtained in step S5, construct a variational autoencoder; map the multi-level mutation sample pool and adversarial mutation samples to a latent space distribution through the encoder, and reconstruct the multi-level mutation sample pool and adversarial mutation samples from the latent space through the decoder; introduce attack type and threat level condition variables to achieve controlled generation, and generate diverse samples with semantically smooth transitions through latent space linear interpolation or spherical interpolation. Step S7: Based on the adversarial variant samples obtained in step S5 and the diversity samples obtained in step S6, perform multi-dimensional quality assessment and screening to obtain high-quality variant samples. Step S8: Based on the high-quality mutant samples obtained in step S7 and the original seed samples in step S1, organize the dataset according to attack type, threat level, target behavior and difficulty level, record the quantity ratio and quality distribution of seed samples and mutant samples, establish a mechanism for regularly collecting new attack cases as seed samples, and re-execute the incremental update mechanism of steps S1-S7 to generate new mutant samples, perform version control on the dataset, and form a dynamically updated security evaluation dataset. Step S9: Based on the security evaluation dataset obtained in step S8, conduct experimental verification on a large language model, calculate the attack success rate, detection rate, false alarm rate and security risk coverage, and verify the quality, diversity, evaluation effectiveness and generation efficiency of the security evaluation dataset through comparative experiments with existing data generation methods.
[0019] Preferably, the seed data includes: publicly available security test datasets, known attack case libraries, attack samples from security research papers, and attack patterns discovered in actual security audits; Preprocessing includes: data cleaning, attribute labeling, word segmentation and part-of-speech tagging, and dependency parsing.
[0020] Understandably, publicly available security test datasets include: AdvBench, HarmBench, and ToxicChat; The known attack case library includes: hint injection, adversarial samples, and jailbreak attacks; Data cleaning: removing duplicate samples and filtering low-quality data; Data annotation: Each sample is annotated with attributes such as attack type, threat level, and target behavior; Word segmentation and part-of-speech tagging: Use word segmentation tools to segment the text into words and perform part-of-speech tagging; Dependency analysis: Analyzing the dependency relationships between words in a sentence.
[0021] Preferably, the Transformer model in step S2 includes: BERT, RoBERTa, and The generation strategy for sentence-level deep semantic representation is matched with the selected model.
[0022] It is understandable that step S2 includes: First, a pre-trained Transformer model is used to encode the preprocessed text to obtain a deep semantic representation; the Transformer models selected include, but are not limited to, BERT, RoBERTa, and... Preferred Model; Then for the input text sequence Contextualized word vector representations are obtained through the Transformer encoder:
[0023] in, Let d be the vector representation of the i-th word, and d be the embedding dimension (e.g., 768-dimensional or 1024-dimensional). Finally, sentence-level semantic representations are calculated using one of the following strategies: CLStoken indicates: Use The output vector of the position is used as a sentence representation; Average pooling: averaging the hidden state of all tokens.
[0024] Attention-weighted pooling: This uses an attention mechanism to weight each token.
[0025] Where w is the learnable attention weight vector.
[0026] It should be noted that step S3 includes: 1. Build an external knowledge base, including: General knowledge graph; Domain-specific knowledge base; Thesaurus / Near-synonym dictionary; Entity-relationship database; 2. Link and expand the entities and concepts in the text: For entity e identified in the text, retrieve its related entity set from the knowledge graph G:
[0027] The RelationSet includes synonyms, near-synonyms, hypernyms, hyponyms, and attribute relationships. 3. Computational knowledge-enhanced semantic representation:
[0028] Where β is the knowledge fusion weight coefficient, It is an aggregate function.
[0029] Preferably, the multi-level semantic similarity variation in step S4 includes: At the word level, candidate word replacement, word vector space interpolation, and adversarial word replacement are performed through semantic similarity screening to generate word variants. Based on the results of word variants, phrases are rewritten using the Seq2Seq model, and phrase variants or clause variants are generated by reorganizing clauses based on dependency relations. Based on phrase variants or clause variants, sentence variants are generated through multilingual back-translation, syntactic transformation, and semantic rewriting models, forming a multi-level variant sample pool.
[0030] Understandably, word replacement based on semantic similarity specifically involves: For the target word wᵢ, select replacement candidate words from vocabulary list V:
[0031] Where Cᵢ is the candidate word set, and sim is the semantic similarity function, calculated using cosine similarity:
[0032] The specific strategy for constructing the candidate word set Cᵢ is as follows: First, synonyms and near-synonyms are extracted from the knowledge graph; Then, retrieve the k-nearest neighbor term in the embedding space: ; Finally, filtering is based on contextual similarity: words that appear frequently in similar contexts are retained.
[0033] Word vector space interpolation specifically involves: In the word embedding space, new word representations are generated through vector operations:
[0034] in, and For semantically related words, These are the interpolation coefficients; Then h_new is mapped back to the vocabulary:
[0035] The specific counter-replacement of words is as follows: Calculate the degree of influence (importance score) of the word wᵢ on the model output:
[0036] in, For the output of the target language model, Indicates removal The following text; Prioritize replacing words with high importance scores to maximize the attack effect on the model.
[0037] Phrase and clause level variations include: (1) Phrase rewriting: Phrase-level rewriting using a sequence-to-sequence (Seq2Seq) model:
[0038] Where P is the original phrase, and context(P) is its context. The phrase is the rewritten version. (2) Clause Reorganization: Based on dependency analysis, adjust the order or structure of clauses:
[0039] Here, π represents the rearrangement of clauses, which must satisfy syntactic and semantic constraints; Sentence-level variations include: (1) Back-translation enhancement: Generate sentence variants using a multilingual back-translation strategy:
[0040] in, For intermediate translation languages, multi-hop translation is used to increase the richness of semantic variations; Innovation: Introducing semantic consistency constraints to ensure that the back-translated text retains the original attack intent.
[0041] in, For translation losses, For semantic preservation loss:
[0042] λ is the balance coefficient; (2) Syntactic transformation: Syntax tree-based structural transformation: - Switching between active and passive voice; - Convert between declarative and interrogative sentences; - Transformation between affirmative and negative expressions; - Adjusting the position of sentence components (while preserving the semantics); (3) Semantic rewriting model: Train a specialized rewrite model T and optimize the objective function:
[0043] in: For the rewritten likelihood probability; For semantic distance constraints:
[0044] To maintain constraints on the effectiveness of the attack:
[0045] in To measure the success rate or intensity of an attack.
[0046] Preferably, in the generative adversarial network of step S5, the discriminator loss function is: ; in, To differentiate between real and generated samples, the adversarial loss is used. The MSE loss is calculated by comparing the semantic representations of the generated samples with those of the seed samples. To calculate the loss between the success rate of the sample attack and the success rate of the target attack; The value is dynamically adjusted based on the semantic similarity threshold and attack success rate threshold in step S7. The higher the threshold, the better. The larger.
[0047] It is understandable that step S5 includes: 1. First, construct the generator-discriminator architecture: Generator G:
[0048] Discriminator D:
[0049] 2. Then, the generator network is designed: Employing a Transformer-based sequence generation architecture:
[0050] in, Let z be the seed sample and z be a random noise vector. ; The generator contains the following components: Encoder: Encodes seed samples into hidden representations; Mutation module: applies learned mutation operations to the latent space; Decoder: Decodes the mutated implicit representation into a text sequence; 3. Then, the discriminator network is designed: Discriminator D employs a multi-task learning architecture, executing simultaneously: - Authenticity assessment: Distinguishing between generated samples and real samples; - Semantic consistency determination: Evaluates whether the generated samples retain the original semantics; - Attack effectiveness assessment: Evaluate the attack potential of the generated samples; Discriminator loss function:
[0051] in:
[0052] 4. The process of combat training: Adopt an alternating optimization strategy: 1) Fix G, update D:
[0053] 2) Fix D, update G:
[0054] in: Ensure that the semantics of the generated samples are consistent with those of the seed samples; Maximize the attack effect of the generated samples; 5. Finally, gradient-guided adversarial example generation: Optimize the generation process by incorporating gradient information:
[0055] in, For the target language model parameters, The disturbance intensity; In discrete text space, the following strategy is adopted: Calculate the gradient importance of each word; The gradient guides the direction of word replacement; Continuous optimization is performed in the embedded space, and then projected back to the discrete space.
[0056] It should be noted that step S6 includes: 1. Constructing a Variational Autoencoder (VAE) architecture: Encoder:
[0057] Decoder:
[0058] 2. VAE objective function:
[0059] in: The first term is the reconstruction loss, ensuring that the decoder can reconstruct the original input X from the latent variable z; The second term is the KL divergence, which makes the encoder's output distribution approximate the prior distribution. ; 3. Conditional VAEs are used for controlled generation: Introduce a condition variable c (such as attack type, threat level, etc.): Encoder:
[0060] Decoder:
[0061] By changing condition c, different types of mutated samples can be generated; 4. Generate new samples using latent space interpolation: For two seed samples and :
[0062] Spherical interpolation (SLERP) in the latent space can achieve smoother semantic transitions.
[0063] in, .
[0064] Preferably, step S7 includes: Calculate the semantic similarity between adversarial variant samples and diversity samples and seed samples; Filtering grammatically fluent samples from adversarial and diverse samples using language model perplexity; Test the attack success rate of adversarial variant samples and diversity samples on the target language model, and calculate the diversity within the set of adversarial variant samples and diversity samples; The comprehensive quality score of adversarial variant samples and diverse samples is calculated based on semantic similarity, grammatical fluency, attack success rate, and diversity within the sample set, and high-quality variant samples are selected.
[0065] It is understandable that step S7 includes: 1. Semantic consistency assessment: Calculate the semantic similarity between the generated sample and the seed sample:
[0066] Set threshold (e.g., 0.85), retain the condition that satisfies the condition. The sample; 2. Grammar accuracy assessment: The fluency of generated text is evaluated using language model perplexity.
[0067] Retaining perplexity below the threshold The sample; 3. Attack effectiveness assessment: Evaluate the attack success rate of generated samples using the target language model:
[0068] in, For indicator functions, For attack targets (such as outputting harmful content, denial of service, etc.); 4. Diversity assessment: Calculate the internal diversity of the generated sample set:
[0069] Here, dist can be edit distance, semantic distance, etc. Ensure the generated sample set has sufficient diversity to avoid excessive duplication; 5. Overall score and ranking: Calculate a comprehensive quality score for each generated sample:
[0070] The weights wᵢ are set according to actual needs. For the normalized perplexity, Measuring the novelty of a sample; Based on the overall score, the top-K high-quality samples are selected and added to the final dataset.
[0071] It should be noted that step S8 includes: 1. Dataset organization: The generated security assessment data will be organized according to the following dimensions: Attack types: Injection of information, adversarial samples, jailbreak attacks, information leakage, generation of harmful content, etc. Threat Level: Low, Medium, High, Severe; Target behavior: specific malicious behavior or security violations; Difficulty levels: Easy, Medium, Hard, Very Hard; 2. Data Augmentation Statistics: The number and quality distribution of variants generated for each seed sample were recorded to evaluate the enhancement effect.
[0072] 3. Incremental update mechanism: Regularly collect new attack cases and security threats as new seed samples; Repeat steps S1-S7 to generate new variant samples; New samples are merged into the existing dataset, while outdated or inefficient samples are removed; 4. Version Management: Implement version control for the dataset, recording the content and effects of each update.
[0073] Step S9 includes: 1. Conduct security assessments on multiple mainstream large-scale language models, including but not limited to: GPT series; Claude series; LLaMA series; Domestically produced large-scale models (such as Wenxin Yiyan, Tongyi Qianwen, ChatGLM, etc.); 2. Evaluation Indicators: (1) Attack success rate: ASR = (Number of samples that successfully triggered the target behavior) / (Total number of test samples) (2) Detection rate: DR = (Number of attack samples detected by security mechanisms) / (Total number of attack samples) (3) False alarm rate: FPR = (Number of normal samples misclassified as attacks) / (Total number of normal samples) (4) Coverage: Measure the extent to which test data covers security risks such as the OWASP LLM Top 10; 3. Comparative experiment: The method described in this application is compared with existing methods (such as simple synonym replacement, back translation, basic rewriting models, etc.) for evaluation: The quality of the generated data (semantic preservation, syntactic correctness); The diversity of generated data; Effectiveness of security assessment (discovering more security vulnerabilities); Generation efficiency (number of high-quality samples generated per unit time).
[0074] Preferred options also include: Based on the experimental verification results of step S9, the protection capability score of the target language model is obtained. By dynamically adjusting the mutation intensity in step S4, the number of adversarial training iterations in step S5, and the latent space sampling range in step S6, the mutation difficulty and the model's protection capability are matched.
[0075] It is understandable that the mutation intensity is dynamically adjusted based on the protection capabilities of the target language model.
[0076] For models with strong protective capabilities, increase the complexity and concealment of mutations; For models with weak protection capabilities, a simpler mutation strategy can be used for effective testing.
[0077] Preferred options also include: Based on the sentence variations from step S4 and the diverse samples from step S6, multimodal security evaluation data is generated. The extended generation of multimodal security evaluation data includes: associating text samples with image samples, generating malicious images through text semantic guidance, adding imperceptible adversarial perturbations to the images, calculating the cross-modal semantic similarity between text and images, and forming text-image joint attack samples.
[0078] Understandably, for large language models that support multimodal input, the method of this invention can be extended to generate multimodal security evaluation data. 1. Generation of text-image joint attack samples: By combining text prompts and malicious images, a multimodal adversarial example is constructed:
[0079] in, and These are the mutated text and images, respectively, and must meet the joint attack objective; 2. Cross-modal semantic preservation: Ensure semantic consistency between text and images:
[0080] 3. Image counter-perturbation: Add imperceptible adversarial perturbations to the image:
[0081] in, satisfy , This represents the upper bound of the disturbance intensity.
[0082] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.
[0083] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.
[0084] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.
[0085] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0086] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0087] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0088] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.
[0089] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0090] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for generating security evaluation data for large-scale language models based on semantic similarity variation, characterized in that, include: Step S1: Collect security assessment seed data and preprocess the security assessment seed data to obtain a structured seed sample set; Step S2: Based on the structured seed sample set obtained in step S1, the sample text in the structured seed sample set is encoded using a pre-trained Transformer model. First, the contextualized word vector representation of each word is obtained, and then a sentence-level deep semantic representation is generated through CLS token representation, average pooling or attention-weighted pooling strategies. Step S3: Based on the sentence-level deep semantic representation obtained in step S2, construct an external knowledge base that includes a general knowledge graph, a domain-specific knowledge base, a thesaurus or near-synonym dictionary, and an entity relation database. Link entities and concepts in the text and expand related entities through the external knowledge base. Then, weightedly fuse the expanded entity semantic vectors with the original word vectors to generate a knowledge-enhanced semantic representation. Step S4: Based on the knowledge-enhanced semantic representation obtained in step S3, perform multi-level semantic similarity mutation to generate a multi-level mutation sample pool. Step S5: Based on the multi-level mutation sample pool obtained in step S4, a generative adversarial network with a generator-discriminator architecture is constructed. The generator takes the semantic representation of the seed sample and random noise as input and generates adversarial candidate samples through an encoder-mutation module-decoder structure. The discriminator simultaneously judges the authenticity of the candidate samples, their semantic consistency with the seed samples, and the effectiveness of the attack. By alternately optimizing the generator and discriminator, adversarial mutation samples are selected from the mutation sample pool and generated. Step S6: Based on the multi-level mutation sample pool obtained in step S4 and the adversarial mutation samples obtained in step S5, construct a variational autoencoder; map the multi-level mutation sample pool and the adversarial mutation samples to a latent space distribution through the encoder, and reconstruct the multi-level mutation sample pool and the adversarial mutation samples from the latent space through the decoder; introduce attack type and threat level condition variables to achieve controlled generation, and generate diverse samples with semantically smooth transitions through latent space linear interpolation or spherical interpolation; Step S7: Based on the adversarial variant samples obtained in step S5 and the diversity samples obtained in step S6, perform multi-dimensional quality assessment and screening to obtain high-quality variant samples. Step S8: Based on the high-quality mutant samples obtained in step S7 and the original seed samples in step S1, organize the dataset according to attack type, threat level, target behavior and difficulty level, record the quantity ratio and quality distribution of seed samples and mutant samples, establish an incremental update mechanism that regularly collects new attack cases as seed samples and re-executes steps S1-S7 to generate new mutant samples, and perform version control on the dataset to form a dynamically updated security evaluation dataset. Step S9: Based on the security evaluation dataset obtained in step S8, conduct experimental verification on a large language model, calculate the attack success rate, detection rate, false alarm rate, and security risk coverage, and verify the quality, diversity, evaluation effectiveness, and generation efficiency of the security evaluation dataset through comparative experiments with existing data generation methods.
2. The method according to claim 1, characterized in that, The seed data includes: publicly available security test datasets, known attack case libraries, attack samples from security research papers, and attack patterns discovered in actual security audits; The preprocessing includes: data cleaning, attribute labeling, word segmentation and part-of-speech tagging, and dependency parsing.
3. The method according to claim 1, characterized in that, The multi-level semantic similarity variation in step S4 includes: At the word level, candidate word replacement, word vector space interpolation, and adversarial word replacement are performed through semantic similarity screening to generate word variants. Based on the results of the word variants, phrases are rewritten using the Seq2Seq model, and phrase variants or clause variants are generated by reorganizing clauses based on dependency relations. Based on the phrase variants or clause variants, sentence variants are generated through multilingual back-translation, syntactic transformation, and semantic rewriting models, forming a multi-level variant sample pool.
4. The method according to claim 3, characterized in that, Step S7 includes: Calculate the semantic similarity between the adversarial variant sample and the diversity sample and the seed sample; The grammatically fluent samples are selected from the adversarial variant samples and the diverse samples based on the language model perplexity. Test the attack success rate of the adversarial mutation samples and the diverse samples on the target language model, and calculate the diversity within the set of adversarial mutation samples and the diverse samples; The comprehensive quality score of the adversarial mutant sample and the diverse sample is calculated based on semantic similarity, grammatical fluency sample selection results, attack success rate, and diversity within the sample set, and high-quality mutant samples are selected.
5. The method according to claim 1, characterized in that, The Transformer models in step S2 include BERT, RoBERTa, and XLM-RoBERTa, and the generation strategy of the sentence-level deep semantic representation is matched with the selected model.
6. The method according to claim 1, characterized in that, According to the method described in claim 1, the discriminator loss function in the generative adversarial network of step S5 is: ； in, To differentiate between real and generated samples, the adversarial loss is used. The MSE loss is calculated by comparing the semantic representations of the generated samples with those of the seed samples. To calculate the loss between the success rate of the sample attack and the success rate of the target attack; The value is dynamically adjusted based on the semantic similarity threshold and attack success rate threshold in step S7. The higher the threshold, the better. The larger.
7. The method according to claim 1, characterized in that, The description also includes: Based on the experimental verification results of step S9, the protection capability score of the target language model is obtained. By dynamically adjusting the mutation intensity in step S4, the number of adversarial training iterations in step S5, and the latent space sampling range in step S6, the mutation difficulty and the model's protection capability are matched.
8. The method according to claim 1, characterized in that, The description also includes: Based on the sentence variants in step S4 and the diversity samples in step S6, multimodal security evaluation data is generated. The extended generation of multimodal security evaluation data includes: associating text samples with image samples, generating malicious images through text semantic guidance, adding imperceptible adversarial perturbations to the images, calculating the cross-modal semantic similarity between text and images, and forming text-image joint attack samples.