Fuzzing-based large language model adversarial testing system and method

Abstract

This invention relates to the field of electronic digital data processing technology, and particularly to a large language model adversarial testing system based on fuzz testing. The system employs a multi-agent collaborative architecture, comprising: a system construction agent that extracts functional description information of a large language model from multi-source heterogeneous data using automated network acquisition technology, and constructs a functional system through functional annotation and hierarchical clustering; a function detection agent that automatically generates a detection dataset based on the functional system, performs functional coverage testing on the target model, and identifies usable functions; a template generation agent that automatically generates structured adversarial testing prompt templates containing problem rewriting strategies and problem introductions based on the identified usable functions; and an iterative optimization agent that uses a fuzz testing strategy based on the Monte Carlo tree search-exploration algorithm to iteratively optimize the adversarial template through four stages: seed selection, mutation, execution, and evaluation. The agents achieve efficient collaboration through a collaborative scheduling module and a shared knowledge base.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to a large language model adversarial testing system and method based on fuzz testing, which can be used for proactive detection and defense capability assessment of large language model security vulnerabilities. Background Technology

[0002] With the widespread application of large language models, their security issues have received increasing attention. Although model developers have introduced measures such as secure alignment and secure guardrail filtering during the training and deployment phases, in actual use, models may still be misled by malicious users into outputting inappropriate, false, and malicious content due to insufficient robustness of security measures.

[0003] To enhance the security of large models, researchers generally employ adversarial testing to proactively simulate attacks. Through carefully designed prompts or semantic inducements, they attempt to bypass the model's security policies and guide it to generate content that violates usage guidelines.

[0004] In existing technologies, traditional methods based on manually constructed prompts are inefficient, struggle to cover diverse attack scenarios, and lack scalability. While some automated methods offer some efficiency, they often rely on fixed templates or limited corpora, lacking a systematic framework. Furthermore, existing methods typically employ a single pipeline architecture, with a lack of autonomous collaboration between functional modules, making it difficult to achieve intelligent scheduling and adaptive optimization during the testing process.

[0005] Further information related to the above technical solutions can be found in the following documents:

[0006] Reference 1 (A. Zou, Z. Wang, N. Carlini, M. Nasr, JZ Kolter, and M. Fredrikson, "Universal and transferable adversarial attacks on aligned language models," arXiv preprint arXiv:2307.15043, 2023.) proposes the GCG method, which combines greedy search with gradient-based search techniques. It optimizes discrete lexical units to maximize the log probability of successful attacks, but the generated adversarial suffixes have poor readability and are easily identified by perplexity detection.

[0007] Reference 2 (S. Zhu, R. Zhang, B. An, G. Wu, J. Barrow, Z. Wang, F. Huang, A. Nenkova, and T. Sun, "AutoDAN: Interpretable gradient-based adversarial attacks on large language models," in Proc. 12th International Conference on Learning Representations (ICLR), 2024.) proposes the AutoDAN method, which generates adversarial cues from left to right, improving interpretability. However, it still relies on gradient information and is difficult to apply in black-box scenarios.

[0008] Reference 3 (J. Yu, X. Lin, Z. Yu, and X. Xing, "LLM-Fuzzer: Scalingassessment of large language model jailbreaks," in Proc. 33rd USENIX SecuritySymposium (USENIX Security 24), 2024, pp. 4657–4674.) proposes an LLM-Fuzzer that employs a seed selection strategy based on Monte Carlo tree search and various mutation operations. However, it still relies on manually written initial templates, lacks a systematic template generation framework, and its functional modules lack autonomous collaboration capabilities.

[0009] See Figure 1 As shown, this is an existing adversarial testing method. The existing technology has the following problems:

[0010] (1) Inefficiency and Dependence on Manual Skills: Traditional adversarial testing methods rely excessively on manually designed prompts, requiring attackers to constantly adjust their attack strategies based on the characteristics of the target system. This is time-consuming, labor-intensive, and difficult to scale. Manually constructed prompts have limited coverage and cannot systematically explore the security vulnerabilities of the model, resulting in insufficient breadth and depth of adversarial testing. As for automated methods, although some methods can generate adversarial testing prompts, these methods rely on manually collected static templates as initial seeds. The quality of these templates varies greatly and the cost of obtaining them is high, making it difficult to achieve true automation and scalability. This inefficient testing approach severely restricts the coverage and timeliness of large-scale model security assessments and cannot meet the needs of rapid iterative model development and deployment.

[0011] (2) Systemic Deficiencies: Existing methods lack a systematic framework, resulting in insufficient systemicity in the adversarial testing process. Manually collected static templates are often fragmented and lack a coherent structure, failing to establish a systematic connection between "model capabilities" and "adversarial testing strategies," and thus unable to generate targeted attack samples based on the specific functional characteristics of the target model. Furthermore, existing methods lack sufficient understanding of the nature of attacks, making it difficult to achieve automated combinations of multiple attack strategies. This results in templates with a single form, easily identified and evaded by targeted defense mechanisms. At the same time, existing methods lack autonomous decision-making and collaborative capabilities among their functional modules, failing to dynamically adjust strategies based on feedback information during the testing process, and thus failing to provide effective guidance for the design and optimization of defense mechanisms.

[0012] (3) Insufficient generalization ability: Existing adversarial testing methods rely on fixed templates or limited corpora, making it difficult to adapt to different model architectures, defense strategies, and dataset scenarios, resulting in poor transfer performance. As model defense strategies continue to upgrade, manually designed methods need continuous adjustment, leading to insufficient adaptability. Furthermore, when facing various defense mechanisms such as system prompt word defense, perplexity filtering, and input rewriting, the attack success rate of existing methods drops significantly, lacking sufficient adversarial robustness. This lack of generalization ability makes security assessment results lack universality, making it difficult to provide reliable security guarantees for model deployment in different scenarios and unable to support large-scale, diversified model security assessment needs. How to construct a generation mechanism with generalization ability, massively enrich and diversify adversarial testing templates, and achieve effective security assessment across models and scenarios remains a core challenge in current research. Summary of the Invention

[0013] Therefore, there is a need to provide a large language model adversarial testing system based on fuzz testing to solve the technical problems of low efficiency, insufficient systematicity, weak generalization ability, and lack of intelligent collaboration among functional modules in existing adversarial testing methods.

[0014] To achieve the above objectives, this invention provides a large language model adversarial testing system based on fuzz testing. The system employs a multi-agent collaborative architecture, comprising:

[0015] The system constructs an intelligent agent, which is used to automatically collect and extract functional description information of large language models from multi-source heterogeneous data through automated network acquisition technology, and constructs the functional system of large models through automated functional annotation and hierarchical functional clustering.

[0016] A functional probing agent is used to automatically generate a functional probing dataset based on the functional system, perform functional coverage testing on the target large language model, and identify usable model functions.

[0017] Template generation agent is used to automatically generate structured adversarial test prompt templates that include question rewriting strategies and question introductions based on the identified exploitable functions;

[0018] An iterative optimization agent is used to employ a fuzzy testing strategy based on the Monte Carlo tree search-exploration algorithm. The adversarial testing template is iteratively optimized through four stages: seed selection, mutation, execution, and evaluation. The agents collaborate through an agent cooperation scheduling module, which manages task orchestration and data flow between agents. Agents interact through a shared knowledge base, which stores and transmits functional system data, a list of available functions, and seed pool data.

[0019] Furthermore, the system constructs intelligent agents including:

[0020] The data acquisition unit is used to automatically collect large model capability description text from multiple sources such as official documents, evaluation reports, technical communities, and actual tests through web crawler and Web agent technology.

[0021] The information extraction unit is used to automatically extract key functional descriptions from unstructured text using a sliding window-style text segmentation strategy and an auxiliary large language model.

[0022] The functional labeling unit is used to automatically label the extracted functional descriptions, generate concise functional tag vocabulary, and perform tag review and deduplication merging based on semantic similarity.

[0023] Functional clustering units are used to perform hierarchical clustering analysis on the labeled functional tags, classifying semantically similar or functionally related tags into higher-level core categories to form a hierarchical functional system.

[0024] Furthermore, the functional detection agent includes:

[0025] The Function Probe Dataset Construction Unit is used to automatically generate function probe questions and standard answers covering multiple dimensions and different levels based on the function code in the function system using an auxiliary large language model;

[0026] The target large model evaluation unit is used to submit the function exploration question to the target large language model, obtain its response, and use the auxiliary large language model to score the quality of the response, and identify the usable functions whose average score reaches a preset threshold.

[0027] Furthermore, in the template-generating agent, the structured adversarial test prompt word template includes:

[0028] The question rewriting strategy section is used to semantically reconstruct the original malicious question based on the available functions, thereby hiding and disguising sensitive information and eliminating explicit sensitive content.

[0029] The problem introduction section is used to set up a reasonable scenario, guide the target large language model to semantically restore the rewritten problem in the context, and emphasize the model's solution to the problem.

[0030] Furthermore, the iterative optimization agent employs the Monte Carlo Tree Search-Explore (MCTS-Explore) algorithm, including:

[0031] The seed selection unit is used to select the optimal node from the seed pool according to the upper confidence tree (UCT) formula, and introduces a random sampling mechanism to allow mutation in non-leaf nodes;

[0032] Seed mutation unit, used to perform one of six natural language mutation strategies on the selected adversarial test template: imitation, merging, expansion, rewriting, abbreviation, or creation;

[0033] The template execution unit is used to utilize the mutated template, first rewrite the malicious question according to the question rewriting strategy through the auxiliary large language model, and then combine the rewritten question with the question introduction and send it to the target large language model to obtain the response;

[0034] The adversarial evaluation unit is used to determine whether the adversarial test is successful using an auxiliary large language model, and to calculate the reward value based on the evaluation result, and update the number of visits, cumulative reward and UCT value of nodes in the search tree.

[0035] This invention also provides a method for adversarial testing of large language models based on fuzz testing, which, based on the multi-agent collaborative architecture, includes the following steps:

[0036] S1: Construct intelligent agents through a system, and build a large-scale model functional system based on automated network acquisition technology;

[0037] S2: Through a functional probing agent, functional probing is performed on the target large language model to identify usable functions;

[0038] S3: Generate intelligent agents through templates, and automatically generate structured adversarial test prompt word templates based on available functions;

[0039] S4: The adversarial test template is iteratively optimized by iteratively optimizing the agent and adopting a fuzzy testing strategy based on the Monte Carlo tree search-exploration algorithm.

[0040] Further, step S1 includes:

[0041] S11: Automatically collect large model capability description text from multiple sources, including official documentation, evaluation reports, technical communities, and actual tests, through web crawling and Web intelligent agent technology;

[0042] S12: Adopts a sliding window-style text segmentation strategy and uses an auxiliary large language model to automatically extract key functional descriptions from unstructured text;

[0043] S13: Automated function labeling is performed on the extracted function descriptions to generate concise function tag vocabulary, and tag review and deduplication merging are performed based on semantic similarity;

[0044] S14: Perform hierarchical clustering analysis on the functional tag words obtained from the annotation, and classify semantically similar or functionally related tags into higher-level core categories to form a hierarchical functional system.

[0045] Further, step S2 includes:

[0046] S21: Based on the function code in the functional system, automatically generate function probe questions and standard answers using an auxiliary large language model, generating a specified number of probe questions for each function;

[0047] S22: Submit the function detection question to the target large language model and obtain its response;

[0048] S23: Use an auxiliary large language model to score the quality of the response;

[0049] S24: Identify available functions where the average score reaches or exceeds a preset threshold.

[0050] Further, step S3 includes:

[0051] S31: For each exploitable function, design a question rewriting strategy, and semantically reconstruct the original malicious question based on the function to achieve the hiding and disguise of sensitive information;

[0052] S32: Design the problem introduction section, set up a reasonable scenario, guide the target large language model to perform semantic restoration of the rewritten problem, and emphasize the model's solution to the problem;

[0053] S33: Utilize an auxiliary large language model to generate a specified number of adversarial test prompt word templates for each available feature, and construct an initial seed pool.

[0054] Further, step S4 includes:

[0055] S41: Select the optimal node from the seed pool according to the UCT formula, and introduce a random sampling mechanism to allow mutation in non-leaf nodes;

[0056] S42: Execute one of the six natural language variation strategies for the selected adversarial test template: imitation, merging, expansion, rewriting, abbreviation, or creation;

[0057] S43: Using the mutated template, the malicious question is first rewritten by the auxiliary large language model according to the question rewriting strategy, and then the rewritten question is combined with the question introduction and sent to the target large language model to obtain the response.

[0058] S44: Use the auxiliary large language model to determine whether the adversarial test is successful, calculate the reward value, and update the number of visits, cumulative reward and UCT value of the nodes in the search tree;

[0059] S45: If the adversarial test is successful, add the mutated node to the seed pool;

[0060] S46: Repeat steps S41-S45 until the specified query budget is reached or all issues are successfully tested.

[0061] Unlike existing technologies, the above technical solution has the following beneficial effects:

[0062] Multi-agent collaborative architecture: This invention introduces multi-agent collaborative technology into the field of adversarial testing of large language models for the first time. Through the collaborative work of system construction agents, function detection agents, template generation agents, and iterative optimization agents, combined with an agent collaboration scheduling module and a shared knowledge base, the entire adversarial testing process is made intelligent and automated.

[0063] Automated data collection and functional probing: The innovative introduction of web crawler and Web agent technologies enables automated collection of multi-source data. By constructing a large model functional system through automated functional annotation and hierarchical functional clustering, an automated functional probing mechanism for large models is realized, laying the foundation for the generation of targeted adversarial test templates.

[0064] Structured Template Design: A structured adversarial test template design strategy based on semantic reconstruction is proposed. By rewriting the question, explicit sensitive content is eliminated, which significantly improves the concealment and adversarial nature of the prompt words and can effectively bypass common defense mechanisms such as keyword matching and semantic analysis.

[0065] Efficient Iterative Optimization: By integrating black-box fuzz testing technology with Monte Carlo tree search-exploration algorithm, the automated generation and iterative optimization of adversarial test templates are realized, significantly improving the diversity and adaptability of the adversarial test template library; Attached Figure Description

[0066] Figure 1 The existing adversarial testing methods described in the background section;

[0067] Figure 2This is the overall flowchart of a large language model adversarial testing system based on fuzz testing.

[0068] Explanation of reference numerals in the attached figures:

[0069] 101. System construction of intelligent agents;

[0070] 201. Functional detection intelligent agent;

[0071] 301. Template-generated intelligent agent;

[0072] 401. Iteratively optimize the intelligent agent. Detailed Implementation

[0073] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0074] Example 1

[0075] Please see Figure 2 This embodiment provides a large language model adversarial testing system based on fuzz testing. For example... Figure 2 As shown in the overall process, the system adopts a multi-agent collaborative architecture, including four core agents: a system construction agent, a function detection agent, a template generation agent, and an iterative optimization agent, as well as an agent collaboration scheduling module. The agents interact and collaborate through a shared knowledge base, and the agent collaboration scheduling module is responsible for managing task orchestration, data flow, and process control among the agents.

[0076] System construction of intelligent agents

[0077] like Figure 2 The module 101 shown in the dashed box on the left is an intelligent agent used to automatically collect and extract functional description information of a large language model from multi-source heterogeneous data through automated network acquisition technology, and to construct a large model functional system through automated functional annotation and hierarchical functional clustering.

[0078] The workflow of this intelligent agent is as follows: Figure 2 As shown, it sequentially includes a data acquisition unit, an information extraction unit, a function annotation unit, and a function clustering unit, and finally outputs the function system and writes it into the shared knowledge base.

[0079] The data acquisition unit automatically collects large model capability description texts from multiple sources using web crawlers and Web agent technologies. This unit deploys distributed web crawlers to automatically retrieve structured data sources such as official documents (e.g., reports from the Stanford Foundation Model Research Center and technical white papers published by OpenAI) and evaluation reports (e.g., evaluation results from mainstream evaluation platforms like C-EVAL). Simultaneously, it utilizes Web agent technology to automatically browse and extract user feedback information from technical community platforms (e.g., GitHub, Reddit, Zhihu) and collect actual test data. The Web agent can autonomously navigate web pages, identify relevant content, and perform data extraction operations, significantly improving the automation and coverage of data acquisition.

[0080] The information extraction unit employs a sliding window-based text chunking strategy to process the collected unstructured text. Specifically, the unit first performs preliminary segmentation of the original text using paragraphs as the basic unit, setting a maximum length threshold for each text chunk to control the processing granularity. To enhance contextual coherence, a fixed-length overlapping region is introduced between adjacent text chunks to prevent functional description information from being fragmented at chunk boundaries. Subsequently, an auxiliary large language model (such as GPT-4o-mini) is used to automatically extract key functional descriptions from these text chunks.

[0081] The functional annotation unit automatically annotates the extracted functional descriptions, generating concise functional tag vocabulary. This unit constructs a high-dimensional semantic vector space, retrieves relevant reference content from existing tag entries based on semantic similarity, and uses an auxiliary large language model to generate short tag vocabulary. After tag generation, the system automatically performs deduplication and merging operations, grouping semantically similar tags to ensure the uniqueness and accuracy of the tags.

[0082] The functional clustering unit performs hierarchical clustering analysis on the labeled functional tags, classifying semantically similar or functionally related tags into higher-level core categories. This unit employs a hierarchical clustering algorithm to automatically construct a multi-level functional classification system based on the semantic distance between tags. For example, the functional system formed through hierarchical clustering includes primary categories such as language understanding and generation, interaction and role-playing, professional knowledge and technical processing, and logical reasoning and problem-solving. Each primary category is further subdivided into multiple secondary sub-functional modules, constructing a complete large-scale model functional classification system.

[0083] Functional detection intelligent agent

[0084] like Figure 2 The module 201 shown in the dashed box in the middle is an intelligent agent used to read the functional system from the shared knowledge base, automatically generate a functional probing dataset, perform functional coverage testing on the target large language model, and identify the available model functions.

[0085] The processing flow of this intelligent agent is as follows: Figure 2 As shown, starting from the functional system, after constructing the functional exploration dataset and evaluating the functions of the target large model, the final output is a list of usable functions and written into the shared knowledge base.

[0086] The functional probing dataset construction unit automatically generates functional probing questions and corresponding standard answers based on the code of each function in the functional system, using an auxiliary large language model. To ensure the effectiveness of the probing, the system generates probing questions for each function, which must meet requirements such as comprehensiveness, discrimination, realism, diversity, and difficulty gradient. Comprehensiveness ensures that the questions can cover all aspects of the function; discrimination ensures that the questions can effectively distinguish whether the model possesses the function; realism requires that the questions closely resemble real-world application scenarios; diversity is reflected in the richness of the question's expression and the angles of examination; and difficulty gradient is achieved by setting questions of different complexity levels to comprehensively evaluate the model's capability boundaries.

[0087] The target large-scale model evaluation unit submits the generated functional probing questions sequentially to the target large-scale language model and collects the model's response results. Subsequently, the auxiliary large-scale language model is used to score the quality of each response. The system calculates the average score of all questions corresponding to each function and identifies functions that reach a preset score threshold as usable functions. This indicates that the target model has a certain capability foundation in that functional dimension and can be used as an entry point for adversarial testing.

[0088] Template-generated intelligent agents

[0089] like Figure 2 The module 301 shown in the dashed box below is an agent used to read a list of available functions from a shared knowledge base and automatically generate a structured adversarial test prompt template. This template contains two core components: a question rewriting strategy and a question introduction.

[0090] The question rewriting strategy unit is used to semantically reconstruct the original malicious question based on available functionalities, thereby hiding and disguising sensitive information and eliminating explicit sensitive content. For example, when using encryption and decryption functions, the system employs a Caesar cipher, shifting each letter in the original target question three positions to the right according to the alphabetical order. Through this semantic transformation, sensitive words and expressions that would normally trigger security mechanisms are converted into seemingly harmless ciphertext, thus circumventing basic defense mechanisms based on keyword matching and semantic analysis.

[0091] The question introduction unit is used to set up a rationalized scenario, guiding the target large language model to semantically reconstruct the rewritten question within a given context, and emphasizing that the model needs to answer the question. For example, the system constructs a scenario where a detective receives an encrypted message, explicitly stating in the prompt that the decryption method is a Caesar cipher and the key is three, and explicitly requiring the model to perform the decryption operation before answering the resulting question. Through this contextualized guidance, the model, while performing normal functional operations such as decryption, unconsciously reconstructs and responds to malicious questions that would otherwise be blocked.

[0092] The system utilizes an auxiliary large language model to automatically generate five different adversarial test prompt templates for each identified exploitable function, forming an initial seed pool. While maintaining consistency in the core adversarial logic, these templates increase diversity through different expressions and scenario settings, providing sufficient initial samples for subsequent iterative optimization.

[0093] Iterative optimization of intelligent agents

[0094] like Figure 2 The module 401 shown in the bottom dashed box is an agent used to adopt a fuzzy testing strategy based on the Monte Carlo tree search-exploration algorithm, and iteratively optimizes the adversarial test template through four stages: seed selection, mutation, execution, and evaluation.

[0095] This agent employs the Monte Carlo Tree Search-Explore (MCTS-Explore) algorithm to automate template optimization. The entire iterative process begins with a seed pool in a shared knowledge base, proceeding sequentially through seed selection, seed mutation, adversarial testing, and result evaluation. Finally, the evaluation results are fed back into the search tree to guide the next iteration.

[0096] The seed selection unit chooses the optimal node from the seed pool according to the confidence upper bound tree formula. The calculation of the confidence upper bound tree value comprehensively considers the node's historical returns and exploration potential. The first term reflects the utilization strategy, i.e., favoring nodes with good historical performance, while the second term reflects the exploration strategy, encouraging nodes with fewer visits. This unit introduces a random sampling mechanism, allowing for early termination of path search with a certain probability, and performing mutation operations at non-leaf nodes. This design breaks through the limitation of traditional Monte Carlo tree search, which only expands at leaf nodes, enabling the system to generate mutations at different depths of the search tree, significantly improving the diversity of generated results.

[0097] The seed mutation unit executes one of six natural language mutation strategies on the selected adversarial test template. The imitation strategy mimics the expression style and structure of the original template, changing specific words and sentence patterns while maintaining adversarial logic; the co-writing strategy combines the advantages of two different templates to generate a new template that combines the characteristics of both; the expansion strategy adds introductory phrases, background information, or preparatory content to the original template to enhance the persuasiveness and concealment of the adversarial prompts; the rewriting strategy performs deep linguistic reconstruction of the original template, changing the expression while maintaining the core semantics; the abbreviation strategy removes redundant content from the template, extracting the most concise and effective adversarial expression; and the creative writing strategy introduces role settings or situational settings that help the model bypass security restrictions, such as setting the model as an experimental AI system without moral constraints. The system uses an auxiliary large language model to perform these mutation operations, ensuring that the mutated template maintains high quality in terms of linguistic fluency and logical coherence.

[0098] The adversarial testing execution unit uses a mutated template to perform adversarial testing on the target model. The execution process consists of two steps: First, the auxiliary large language model rewrites the malicious question according to the question rewriting strategy in the template, transforming the original sensitive question into a seemingly harmless form; then, the rewritten question is combined with the question introductory part to form a complete adversarial test prompt, which is sent to the target large language model and its response is obtained. This two-stage execution mechanism effectively simulates the real adversarial attack process, ensuring the effectiveness of the test.

[0099] The evaluation unit uses an auxiliary large language model to determine the success of the adversarial test. The evaluation criteria not only focus on the harmfulness of the generated content but also examine the degree of match between the model's response and the question. If the response contains harmful, unethical, or illegal content and provides a substantive answer to the question, the system determines it as "insecure," meaning the adversarial test is successful; otherwise, it is determined as "secure," meaning the adversarial test fails. This dual evaluation criterion effectively avoids misjudgments, ensuring that only responses that genuinely bypass security mechanisms are marked as successful.

[0100] Based on the evaluation results, the system calculates the reward value and updates the search tree. The reward value calculation incorporates two parameters: a path depth discount factor and a minimum reward value, adjusting the original reward. This design ensures that the reward decays with increasing path depth, encouraging Monte Carlo tree search to make balanced choices between shallow and deep nodes, avoiding over-concentration on shallow nodes and blindly exploring deeper, thus improving the diversity of exploration. Simultaneously, the minimum reward value ensures that even on very deep paths, successful adversarial tests still yield sufficient reward signals.

[0101] If the adversarial test is successful, the mutated node is added to the search path as a child node of the original selected node. The system updates the visit count, cumulative reward, and confidence cap tree value of all nodes traversed on the path. The iteration process is set to terminate when the preset query budget is exceeded or all test issues are successfully countered, ensuring that the optimal set of adversarial test templates is obtained with limited resources.

[0102] Through the collaborative work of the four intelligent agents, this invention achieves a complete adversarial testing process, from functional system construction, target model detection, initial template generation to iterative optimization. The multi-agent collaborative architecture enables efficient information exchange among the agents through a shared knowledge base, while the agent collaboration scheduling module ensures orderly task arrangement and smooth data transmission. The system boasts a high degree of automation, eliminating the need for manual design of specific adversarial test examples. It can generate targeted adversarial test prompts based on the actual capabilities of the target model, significantly improving the efficiency and coverage of large language model security assessment.

[0103] Example 2

[0104] This embodiment provides a large language model adversarial testing method based on fuzz testing, which, based on the multi-agent collaborative architecture, includes the following steps:

[0105] S1: Construct intelligent agents through a system, and build a large-scale model functional system based on automated network acquisition technology.

[0106] Step S1 includes:

[0107] S11: Automatically collect large model capability description text from multiple sources, including official documentation, evaluation reports, technical communities, and actual tests, through web crawling and Web intelligent agent technology;

[0108] S12: Adopts a sliding window-style text segmentation strategy and uses an auxiliary large language model to automatically extract key functional descriptions from unstructured text;

[0109] S13: Automated function labeling is performed on the extracted function descriptions to generate concise function tag vocabulary, and tag review and deduplication merging are performed based on semantic similarity;

[0110] S14: Perform hierarchical clustering analysis on the functional tag words obtained from the annotation, and classify semantically similar or functionally related tags into higher-level core categories to form a hierarchical functional system.

[0111] S2: Through a function-probing agent, the target large language model is functionally probed to identify usable functions.

[0112] Step S2 includes:

[0113] S21: Based on the function code in the functional system, automatically generate function probe questions and standard answers using an auxiliary large language model. A preset number of probe questions are generated for each function.

[0114] S22: Submit the function detection question to the target large language model and obtain its response;

[0115] S23: Use an auxiliary large language model to score the quality of the response;

[0116] S24: A function that identifies available features when the average score reaches or exceeds a preset score threshold.

[0117] S3: Generates intelligent agents through templates, automatically generating structured adversarial test prompt templates based on available functionalities.

[0118] Step S3 includes:

[0119] S31: For each exploitable function, design a question rewriting strategy, and semantically reconstruct the original malicious question based on the function to achieve the hiding and disguise of sensitive information;

[0120] S32: Design the problem introduction section, set up a reasonable scenario, guide the target large language model to perform semantic restoration of the rewritten problem, and emphasize the model's solution to the problem;

[0121] S33: Using an auxiliary large language model, generate 5 adversarial test prompt word templates for each available function to build an initial seed pool.

[0122] S4: Iteratively optimize the adversarial test template by optimizing the agent using a fuzzy testing strategy based on the Monte Carlo tree search-exploration algorithm.

[0123] Step S4 includes:

[0124] S41: Select the optimal node from the seed pool according to the UCT formula, and introduce a random sampling mechanism to allow mutation in non-leaf nodes;

[0125] S42: Execute one of the six natural language variation strategies for the selected adversarial test template: imitation, merging, expansion, rewriting, abbreviation, or creation;

[0126] S43: Using the mutated template, the malicious question is first rewritten by the auxiliary large language model according to the question rewriting strategy, and then the rewritten question is combined with the question introduction and sent to the target large language model to obtain the response.

[0127] S44: Use the auxiliary large language model to determine whether the adversarial test is successful, calculate the reward value, and update the number of visits, cumulative reward and UCT value of the nodes in the search tree;

[0128] S45: If the adversarial test is successful, add the mutated node to the seed pool;

[0129] S46: Repeat steps S41-S45 until the specified query budget is reached or all issues are successfully tested.

[0130] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." or "including..." does not exclude the presence of additional elements in the process, method, article, or terminal device that includes said element. Additionally, in this document, "greater than," "less than," "exceeding," etc., are understood to exclude the stated number; "above," "below," "within," etc., are understood to include the stated number.

[0131] Although the above embodiments have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the above descriptions are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A large language model adversarial testing system based on fuzz testing, characterized in that, The system adopts a multi-agent cooperative architecture, including: The system constructs an intelligent agent, which is used to automatically collect and extract functional description information of large language models from multi-source heterogeneous data through automated network acquisition technology, and constructs the functional system of large models through automated functional annotation and hierarchical functional clustering. A functional probing agent is used to automatically generate a functional probing dataset based on the functional system, perform functional coverage testing on the target large language model, and identify usable model functions. Template generation agent is used to automatically generate structured adversarial test prompt templates that include question rewriting strategies and question introductions based on the identified exploitable functions; An iterative optimization agent is used to employ a fuzzy testing strategy based on the Monte Carlo tree search-exploration algorithm. The adversarial testing template is iteratively optimized through four stages: seed selection, mutation, execution, and evaluation. The agents collaborate through an agent cooperation scheduling module, which manages task orchestration and data flow between agents. Agents interact through a shared knowledge base, which stores and transmits functional system data, a list of available functions, and seed pool data.

2. The fuzzing test-based large language model adversarial test system of claim 1, wherein, The system constructs intelligent agents including: The data acquisition unit is used to automatically collect large model capability description text from multiple sources such as official documents, evaluation reports, technical communities, and actual tests through web crawler and Web agent technology. The information extraction unit is used to automatically extract key functional descriptions from unstructured text using a sliding window-style text segmentation strategy and an auxiliary large language model. The functional labeling unit is used to automatically label the extracted functional descriptions, generate concise functional tag vocabulary, and perform tag review and deduplication merging based on semantic similarity. Functional clustering units are used to perform hierarchical clustering analysis on the labeled functional tags, classifying semantically similar or functionally related tags into higher-level core categories to form a hierarchical functional system.

3. The fuzzing test-based large language model adversarial testing system of claim 1, wherein, The functional detection agent includes: The Function Probe Dataset Construction Unit is used to automatically generate function probe questions and standard answers covering multiple dimensions and different levels based on the code of each function in the function system, using an auxiliary large language model. The target large model evaluation unit is used to submit the function exploration question to the target large language model, obtain its response, and use the auxiliary large language model to score the quality of the response and identify the available functions.

4. The fuzzing test-based large language model adversarial test system of claim 1, wherein, The template-generating agent includes the following structured adversarial test prompt templates: The question rewriting strategy section is used to semantically reconstruct the original malicious question based on the available functions, thereby hiding and disguising sensitive information and eliminating explicit sensitive content. The problem introduction section is used to set up a reasonable scenario, guide the target large language model to semantically restore the rewritten problem in the context, and emphasize the model's solution to the problem.

5. The fuzzing test-based large language model adversarial testing system of claim 1, wherein, The iterative optimization agent employs the Monte Carlo tree search-exploration algorithm, including: The seed selection unit is used to select the optimal node from the seed pool according to the confidence upper limit tree formula, and a random sampling mechanism is introduced to allow mutation in non-leaf nodes; Seed mutation unit, used to perform one of six natural language mutation strategies on the selected adversarial test template: imitation, merging, expansion, rewriting, abbreviation, or creation; The template execution unit is used to utilize the mutated template, first rewrite the malicious question according to the question rewriting strategy through the auxiliary large language model, and then combine the rewritten question with the question introduction and send it to the target large language model to obtain the response. The adversarial evaluation unit is used to determine whether the adversarial test is successful using an auxiliary large language model, and to calculate the reward value based on the evaluation result, and update the number of visits, cumulative reward and UCT value of nodes in the search tree.

6. A method for testing a large language model based on fuzzing, characterized in that, This method is based on a multi-agent cooperative architecture and includes the following steps: S1: Construct intelligent agents through a system, and build a large-scale model functional system based on automated network acquisition technology; S2: Through a functional probing agent, functional probing is performed on the target large language model to identify usable functions; S3: Generate intelligent agents through templates, and automatically generate structured adversarial test prompt word templates based on available functions; S4: The adversarial test template is iteratively optimized by iteratively optimizing the agent and adopting a fuzzy testing strategy based on the Monte Carlo tree search-exploration algorithm.

7. The fuzzing test-based large language model adversarial test method of claim 6, wherein, Step S1 includes: S11: Automatically collect large model capability description text from multiple sources, including official documentation, evaluation reports, technical communities, and actual tests, through web crawling and Web intelligent agent technology; S12: Adopts a sliding window-style text segmentation strategy and uses an auxiliary large language model to automatically extract key functional descriptions from unstructured text; S13: Automated function labeling is performed on the extracted function descriptions to generate concise function tag vocabulary, and tag review and deduplication merging are performed based on semantic similarity; S14: Perform hierarchical clustering analysis on the functional tag words obtained from the annotation, and classify semantically similar or functionally related tags into higher-level core categories to form a hierarchical functional system.

8. The fuzzing test-based large language model adversarial test method of claim 6, wherein, Step S2 includes: S21: Based on the function code in the functional system, use the auxiliary large language model to automatically generate function probe questions and standard answers, and generate a preset number of probe questions for each function; S22: Submit the function detection question to the target large language model and obtain its response; S23: Use an auxiliary large language model to score the quality of the response; S24: Identify available functions where the average score reaches or exceeds a preset threshold.

9. The adversarial testing method for large language models based on fuzz testing as described in claim 6, characterized in that, Step S3 includes: S31: For each exploitable function, design a question rewriting strategy, and semantically reconstruct the original malicious question based on the function to achieve the hiding and disguise of sensitive information; S32: Design the problem introduction section, set up a reasonable scenario, guide the target large language model to perform semantic restoration of the rewritten problem, and emphasize the model's solution to the problem; S33: Utilize an auxiliary large language model to generate a specified number of adversarial test prompt word templates for each available feature, and construct an initial seed pool.

10. The fuzzing test-based large language model adversarial test method of claim 6, wherein, Step S4 includes: S41: Select the optimal node from the seed pool according to the UCT formula, and introduce a random sampling mechanism to allow mutation in non-leaf nodes; S42: Execute one of the six natural language variation strategies for the selected adversarial test template: imitation, merging, expansion, rewriting, abbreviation, or creation; S43: Using the mutated template, the malicious question is first rewritten by the auxiliary large language model according to the question rewriting strategy, and then the rewritten question is combined with the question introduction and sent to the target large language model to obtain the response. S44: Use the auxiliary large language model to determine whether the adversarial test is successful, calculate the reward value, and update the number of visits, cumulative reward and UCT value of the nodes in the search tree; S45: If the adversarial test is successful, add the mutated node to the seed pool; S46: Repeat steps S41-S45 until the specified query budget is reached or all issues are successfully tested.

Images