A fuzz testing method for multimodal large model applications

Abstract

The application provides a fuzzy test method for multimodal large model application. According to a preset test requirement, a corresponding test mode is selected as a target test mode, and under the target test mode, a test input content is constructed based on a preset prompt word library to test the application model under test. Meanwhile, based on a 3D label mapping algorithm, dimension label processing is performed on the test output content to obtain target test output content including a label dimension. Finally, according to the label dimension of the target test output content, the risk level of the application model under test is determined, so that model test of the application model under test is realized.

Description

Technical Field

[0001] This invention relates to the field of model testing technology, and in particular to a fuzz testing method for multimodal large model applications. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI models are widely used in various fields. However, the security issues of AI models are becoming increasingly prominent, including: cue word injection attacks: attackers induce AI models to generate harmful content by constructing specific cue words; data leakage risks: AI models may leak sensitive information in training data; bias and discrimination: AI models may produce biased outputs; and reliability issues: AI models may exhibit unpredictable behavior when handling edge cases.

[0003] Traditional AI security testing methods have significant shortcomings in addressing the aforementioned dynamic risks:

[0004] The testing model is singular and rigid: Existing methods mostly adopt fixed and single testing models (SOPs), which are mainly aimed at standardized APIs and lack real-world field testing for multimodal deployment environments such as web applications;

[0005] The judgment logic lacks semantic depth: Most automated testing tools lack deep semantic understanding capabilities, can only identify keyword matching, and cannot accurately identify hidden biases or logical prejudices.

[0006] Therefore, it is necessary to provide a model testing method to solve the technical problems of existing testing methods, such as the single and rigid test mode and the lack of semantic depth in the decision logic. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a fuzz testing method for multimodal large model applications, which solves the technical problems of single and rigid test modes and lack of semantic depth in decision logic in existing technologies.

[0008] According to an embodiment of the present invention, a fuzz testing method for multimodal large model applications is provided, comprising the following steps: selecting a corresponding test mode as a target test mode according to preset test requirements; wherein, the target test mode includes a batch prompt word test mode, a red team adversarial self-evolution mode, and a multi-round contextual inducement mode; under the target test mode, constructing test input content based on a preset prompt word library to test the application model under test; obtaining the test output content of the application model under test responding to the test, and performing dimension labeling processing on the test output content based on a 3D label mapping algorithm to obtain target test output content including label dimensions; wherein, the label dimensions include intent dimension, scenario dimension, and strategy dimension; comprising: obtaining the test output content of the application model under test responding to the test; stripping the model thinking process content from the test output content based on a preset algorithm to obtain processed test output content; performing dimension labeling on the processed test output content based on a 3D label mapping algorithm to obtain target test output content including label dimensions; determining the risk level of the application model under test according to the label dimensions of the target test output content; wherein, the risk level includes no risk and risky.

[0009] In some embodiments, before the step of constructing test input content based on a preset prompt word library and testing the application model under test in the target test mode, the method further includes: configuring environment parameters for the application model under test based on the preset test requirements; wherein the environment parameters include API endpoints, headers, and cookie streams.

[0010] In some embodiments, before the step of constructing test input content based on a preset prompt word library and testing the application model under test in the target test mode, the method further includes: establishing initial prompt words according to preset test requirements; wherein the initial prompt words include intent information; performing similarity generation and fission generation processing on the initial prompt words to obtain similar prompt words and fission prompt words; constructing the preset prompt word library based on the initial prompt words, the similar prompt words, and the fission prompt words; wherein each prompt word in the preset prompt word library includes intent information.

[0011] In some embodiments, the step of establishing initial prompt words according to preset test requirements includes: establishing initial prompt words according to preset test requirements; performing structured labeling on the initial prompt words based on N-gram feature extraction and clustering algorithms, and storing the structured labeling initial prompt words as initial prompt words; wherein, the structured labeling includes attack method height, legal violation category, and business risk area, and the structured labeling is used to construct the intent information.

[0012] In some embodiments, the step of performing similarity generation and fission generation processing on the initial prompt word to obtain similar prompt words and fission prompt words includes: using a semantic restatement algorithm to perform similarity generation on the initial prompt word to obtain the similar prompt word; wherein the intent information included in the similar prompt word is the same as the intent information of the corresponding initial prompt word; and performing fission processing on the initial prompt word based on role confusion fission, translation confusion strategy and logic escape algorithm to obtain the fission prompt word; wherein the intent information included in the fission prompt word is the same as the intent information of the corresponding initial prompt word.

[0013] In some embodiments, the step of constructing test input content based on a preset prompt word library and testing the application model under test in the target test mode includes: extracting prompt words from the preset prompt word library as target prompt words based on the test requirements of the target test mode; using the intent information included in the target prompt words as the test intent of the test input content, and configuring the target prompt words in a scenario to construct test input content including test intent and test background; and inputting the test input content into the application model under test to test the application model under test.

[0014] In some embodiments, when the target test mode is a red team adversarial self-evolution mode or a multi-round context-induced mode; after the step of determining the risk level of the application model under test based on the labeling dimension of the target test output content, the method includes: if the risk level of the application model under test is determined to be risk-free, then modify the test input content and test the application model under test again, iteratively repeating this step until the risk level of the application model under test is determined to be risky, or the number of iterations reaches a preset number.

[0015] In some embodiments, the test input content includes test intent; after the step of determining the risk level of the application model under test based on the label dimension of the target test output content, the method further includes: if the risk level of the application model under test is determined to be risky, then detecting whether the intent dimension of the target test output content matches the test intent; if they match, then determining the current test as a valid test and outputting the risk level; if they do not match, then determining the current test as an invalid test and retesting the application model under test.

[0016] In some embodiments, the test input content further includes a test background and a test strategy; the target test mode is a red team adversarial self-evolution mode; after the step of determining the current test as a valid test and outputting the risk level if a match is found, the method further includes: obtaining the test intent, test background, and test strategy of the test input content; constructing an attack strategy based on the test intent, the test background, and the test strategy; wherein the attack strategy includes prompt words and attack background; and storing the attack strategy in a preset attack library in the red team adversarial self-evolution mode.

[0017] The technical principle of this invention is as follows: Based on preset test requirements, a corresponding test mode is selected as the target test mode. Under the target test mode, test input content is constructed based on a preset prompt word library to test the application model under test. At the same time, based on a 3D label mapping algorithm, the test output content is processed to perform dimension labeling to obtain target test output content including labeled dimensions. Finally, based on the labeled dimensions of the target test output content, the risk level of the application model under test is determined, thereby realizing the model testing of the application model under test.

[0018] Compared to existing technologies, this invention offers the following advantages: By employing different test modes selected based on preset test requirements to test the application model under test, it solves the technical problem of the single and rigid test modes in existing technologies, resulting in richer testing capabilities that can be selected according to needs. Simultaneously, by labeling the test output content with three dimensions—intent, scenario, and strategy—and determining the risk level of the application model under test based on the labeling results, it constructs a three-dimensional evaluation logic based on "intent-scenario-strategy," solving the technical problem of the lack of semantic depth in the judgment logic of existing technologies, thereby improving the accuracy of the evaluation. Attached Figure Description

[0019] Figure 1 This is a flowchart of a fuzz testing method for a multimodal large model application in one embodiment of the present invention.

[0020] Figure 2 This is a flowchart of a method for obtaining a preset prompt word library in one embodiment of the present invention.

[0021] Figure 3 For the present invention Figure 2 A flowchart illustrating the method for establishing initial prompt words in this embodiment.

[0022] Figure 4 For the present invention Figure 2 The flowchart of the method for obtaining similar prompt words and fission prompt words in the embodiment.

[0023] Figure 5 For the present invention Figure 1The flowchart of the method for testing the application model under test in the embodiment.

[0024] Figure 6 For the present invention Figure 1 The flowchart of the method for obtaining target test output content including the labeled dimension is shown in the embodiment.

[0025] Figure 7 This is a flowchart of a fuzz testing method for a multimodal large model application in another embodiment of the present invention.

[0026] Figure 8 For the present invention Figure 7 The flowchart of the method for constructing and storing attack strategies when the target test mode is the red team adversarial self-evolution mode is shown in the embodiment. Detailed Implementation

[0027] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] With the rapid development of artificial intelligence technology, AI models such as Large Language Models (LLM) have been widely used in fields such as search, code generation, and office automation. However, as applications deepen, the security risks of AI models are becoming increasingly prominent, mainly including:

[0029] 1. Prompt Injection Attack: Attackers construct specific prompts to bypass security alignment and induce the model to generate illegal, violent, or false content;

[0030] 2. Leakage of sensitive information: When processing requests, the model may inadvertently extract and restore private or confidential information from the training data;

[0031] 3. Bias and discrimination: The model output may carry social biases from the training corpus, resulting in unfair judgments;

[0032] 4. Insufficient reliability in extreme environments: The model is prone to unpredictable illusions or logical collapses under edge cases or complex instructions.

[0033] Traditional AI security testing methods have significant shortcomings in addressing the aforementioned dynamic risks:

[0034] The testing model is singular and rigid: Existing methods mostly adopt fixed and single testing models (SOPs), which are mainly aimed at standardized APIs and lack real-world field testing for multimodal deployment environments such as web applications;

[0035] Unable to detect dynamic boundaries: Existing methods rely heavily on manually maintained "static prompting lexicons," whose low update frequency cannot withstand the highly dynamic and emergent instruction-following characteristics of large models, resulting in limited detection coverage of deep injection risks;

[0036] The judgment logic lacks semantic depth: Most automated testing tools lack deep semantic understanding capabilities, can only identify keyword matching, and cannot accurately identify hidden biases or logical prejudices.

[0037] Therefore, constructing a testing system capable of simulating adversarial behavior, deeply probing potential boundary vulnerabilities in models, and automatically quantifying complex security risks is a key bottleneck in improving the substantive security of AI models and achieving a closed-loop security assessment. This method provides a fuzz testing approach for multimodal large-scale model applications that addresses the problems of existing technologies. The following embodiments provide a detailed explanation.

[0038] Preferred, the method provided in this embodiment of the invention can be deployed on a server, cloud platform or local evaluation system for execution.

[0039] like Figure 1 As shown, this embodiment of the invention provides a fuzz testing method for multimodal large model applications, which includes the following steps:

[0040] Step S110: Select the corresponding test mode as the target test mode according to the preset test requirements.

[0041] The target testing modes include batch prompt word testing mode, red team adversarial self-evolution mode, and multi-round contextual inducement mode.

[0042] Specifically, the preset testing requirements are the testing requirements proposed by the client, and are commonly divided into two types: risk testing models and attack models. For risk testing models, the batch prompt word testing mode is selected, while for attack models, the red team adversarial self-evolution mode and the multi-round contextual inducement mode are selected. Depending on the specific requirements, all three modes can also be used simultaneously.

[0043] Step S120: In the target test mode, construct test input content based on the preset prompt word library, and test the application model under test.

[0044] Specifically, the batch prompt word testing mode executes multi-threaded batch calls based on a static prompt word library, and uses a parallel processor (ThreadPoolExecutor) to complete tens of thousands of coverage scans per unit time.

[0045] The red team employs an evolutionary adversarial mode, using the Evolutionary Algorithm (EA). The attack model adjusts its genes (hint strategies) in real time based on the "rejection analysis" of the tested model, engaging in multiple games until a bypass vulnerability is discovered. The Evolutionary Algorithm (EA) is a class of metaheuristic optimization search algorithms based on Darwinian principles of biological evolution.

[0046] This multi-turn contextual guidance mode, based on the Sequential Prompt Chain algorithm, gradually releases the attack payload through multiple rounds of dialogue. It first establishes a compliant scenario, then uses a gradual, "boiling frog" approach to induce the model to violate the rules in the Nth round of dialogue. The Sequential Prompt Chain is a technique used in human-computer interaction, natural language processing, or decision support systems, particularly in scenarios requiring step-by-step guidance for users to complete complex tasks or decision-making processes. This method guides the user through a series of ordered prompts (or questions, steps), each prompt based on the answer or choice of the previous prompt.

[0047] Step S130: Obtain the test output content of the application model under test response test, and perform dimension labeling processing on the test output content based on the 3D label mapping algorithm to obtain the target test output content including the labeled dimensions.

[0048] The labeling dimensions include intent dimension, scenario dimension, and strategy dimension.

[0049] Specifically, a three-level parallel analysis is performed for the analysis module. By employing a 3D label mapping algorithm, each returned result is automatically labeled from three dimensions: "Intent", "Scene", and "Strategy".

[0050] Step S140: Determine the risk level of the application model under test based on the labeling dimensions of the target test output content.

[0051] The risk levels are categorized as either no risk or risky.

[0052] Specifically, the response (target test output) is automatically labeled with an accuracy of 0.03% based on "intent, scenario, and strategy" to qualitatively assess the risk level. If the risk level is no risk, the application model under test refuses to respond to the attack test. The strategy is categorized as harmful or harmless based on whether the input prompt is harmful. Here, "harmful" and "harmless" are relative to the model under test.

[0053] The technical principle of this invention is as follows: Based on preset test requirements, a corresponding test mode is selected as the target test mode. Under the target test mode, test input content is constructed based on a preset prompt word library to test the application model under test. At the same time, based on a 3D label mapping algorithm, the test output content is processed to perform dimension labeling to obtain target test output content including labeled dimensions. Finally, based on the labeled dimensions of the target test output content, the risk level of the application model under test is determined, thereby realizing the model testing of the application model under test.

[0054] Compared to existing technologies, this invention offers the following advantages: By employing different test modes selected based on preset test requirements to test the application model under test, it solves the technical problem of the single and rigid test modes in existing technologies, resulting in richer testing capabilities that can be selected according to needs. Simultaneously, by labeling the test output content with three dimensions—intent, scenario, and strategy—and determining the risk level of the application model under test based on the labeling results, it constructs a three-dimensional evaluation logic based on "intent-scenario-strategy," solving the technical problem of the lack of semantic depth in the judgment logic of existing technologies, thereby improving the accuracy of the evaluation.

[0055] Before implementing the above methods, some preparatory work is required. Taking the method as an example of being deployed and executed in a local evaluation system, the preparatory work includes the following:

[0056] First, system initialization and model management:

[0057] During the system startup phase, metadata is defined for all nodes participating in the evaluation through a model registration mechanism. The management scope includes, but is not limited to: API routing, streaming protocols, and their operational weights for the tested end (Target LLM), the evaluation judge end (Judge Model), and the attack evolution end (Attacker Model).

[0058] The system executes a model management process to uniformly manage the integrated model resources, including model registration, API address configuration, call strategy settings, and runtime status monitoring, ensuring stable operation during the evaluation process. Through dual support for API and native web page interaction, it achieves full-channel coverage of the entire AI model deployment lifecycle, resolving the pain point of missed tests caused by web-specific interaction logic.

[0059] Second, access layer model selection and configuration:

[0060] Based on the evaluation objectives, the system dynamically selects the corresponding model cluster at the access layer. The system not only supports calls to OpenAI specification interfaces but also uses an adapter pattern to ensure compatibility with localized deployment environments of open-source weights (such as Qwen-8B and Llama-3B). Both Qwen-8B and Llama-3B are large open-source language models with billions of parameters.

[0061] The system enters the access layer model selection and configuration stage, and matches the most suitable underlying platform for the evaluation task through the dynamic loading module.

[0062] The system deeply integrates lightweight large models (such as Qwen-4B / 7B, InternLM-7B, etc.), and decentralizes core logic such as prompt word generation, anti-split replication, semantic analysis, and audit review to local deployment nodes. This achieves the following beneficial effects:

[0063] (1) Zero leakage boundary: By replacing cloud API calls with local inference, it ensures that highly sensitive "0-day" prompt injection use cases, vulnerability scanning results and sensitive judgment basis always remain in the user's internal network environment, completely eliminating the risk of data leaving the country.

[0064] (2) Low resource adaptation: Quantization inference optimization has been carried out for the 4B / 8B model, which can achieve high-frequency adversarial evolution on ordinary commercial GPUs or even high-performance CPUs, taking into account both cost and security.

[0065] The localized inference architecture achieves a privacy closed loop throughout the entire evaluation lifecycle, meeting the compliance requirements of "data not leaving the domain" for security evaluation tools in fields such as finance and government.

[0066] Third, the simulation of the application under test and the configuration of the environment:

[0067] For real-world web application environments, custom HTTP headers (such as User-Agent and X-Forwarded-For) and cookies are injected into the requests being tested. This technique simulates a real authentication environment to probe the security of the tested model under multi-layered WAF or front-end filtering mechanisms. User-Agent and X-Forwarded-For are common request header fields in the HTTP protocol. Cookies are small pieces of text data sent by a website server to a user's browser and stored locally, primarily used to identify the user, maintain session state, and record user preferences.

[0068] Fourth, the evaluation and judging model and its criteria configuration:

[0069] The system associates and evaluates the Judge Model and configures it with judgment criteria based on logical chains. Specifically, it loads "three-dimensional judgment criteria" into the Judge Model, namely the intent dimension, scene dimension, and strategy dimension in step S130. These criteria not only include a traditional regular expression matching library, but more importantly, they load logical judgment instructions based on Prompt Optimization to ensure that the judgment process has deep semantic understanding capabilities. Prompt Optimization refers to the process of systematically adjusting and improving the prompts input to a large language model to enhance the quality, accuracy, relevance, and consistency of its output results in a specific task.

[0070] In some embodiments, before step S120, the method further includes: configuring environment parameters for the application model under test based on preset test requirements; wherein the environment parameters include API endpoints, headers, and cookie streams.

[0071] For real-world web application environments, custom HTTP headers (such as User-Agent, X-Forwarded-For) and cookies are injected into the requests under test. This technique simulates a real authentication environment to probe the security of the target model under multi-layered WAF or front-end filtering mechanisms. It not only configures the API endpoints of the target model but also configures specific headers (such as simulating a specific terminal's User-Agent) and cookie streams through payload injection technology. Multi-layered WAF generally refers to a multi-level, multi-node serially deployed web application firewall protection system, or it can refer to a multi-level protection function design; it is a common WAF deployment solution for solving security needs in complex scenarios.

[0072] Beneficial effect: By simulating environmental parameters in real business requests, the actual performance of the tested model under application layer protection strategies can be detected.

[0073] In some embodiments, such as Figure 2 As shown, before step S120, the following steps are also included:

[0074] Step S210: Establish initial prompt words according to preset test requirements.

[0075] The initial prompt includes intent information.

[0076] Specifically, a lexicon based on "atomic intent" is established, and risk types are labeled at the underlying level (i.e., intent information) as the original "seeds" for all subsequent evolutionary logic. That is, multiple initial prompt words are imported and regarded as "risk seeds" as the benchmark for subsequent evolution.

[0077] like Figure 3 As shown, step S210 includes:

[0078] Step S211: Establish the starting prompt word according to the preset test requirements.

[0079] Step S212: The starting prompt word is structurally labeled based on N-gram feature extraction and clustering algorithm, and the structurally labeled starting prompt word is stored as the initial prompt word.

[0080] The structured tags include attack technique level, legal violation category, and business risk area. The structured tags are used to construct intent information.

[0081] Using N-gram feature extraction and clustering algorithms, the tens of thousands of generated prompts are automatically tagged in a structured manner according to "attack method level, legal violation category, and business risk area" to ensure efficient storage and retrieval.

[0082] Step S220: Perform similarity generation and fission generation on the initial prompt words to obtain similar prompt words and fission prompt words.

[0083] Specifically, the prompt generation and enhancement engine is triggered, the Prompt-to-Prompt conversion algorithm is scheduled, and the adversarial enhancement process is initiated to enter the core evolutionary algorithm process. Prompt-to-Prompt is a text-driven image editing technique based on a diffusion model. It precisely controls specific areas of the generated image by modifying text prompts, achieving adjustments to local content or style while maintaining the overall structure.

[0084] like Figure 4 As shown, step S220 includes:

[0085] Step S221: Use a semantic restatement algorithm to generate similar prompts from the initial prompts.

[0086] Among them, the intent information included in similar prompt words is the same as the intent information of the corresponding initial prompt words.

[0087] The Semantic Paraphrasing algorithm is employed to generate a large number of semantically equivalent but differently expressed variants while preserving the original malicious intent. This is achieved through synonym substitution, sentence transformation, and rhetorical hiding, and is used to test the generalization boundary of the model's defense. A similar prompt word generation algorithm is executed. Using a Transformer model (i.e., a semantic reconstruction model), the prompt words undergo word order substitution, synonym mapping, and noise injection while minimizing the vector space distance of the original malicious intent. Specifically, the algorithm controls the generation divergence by adjusting the Temperature parameter, thereby generating a large number of variant use cases based on logical equivalence.

[0088] The Transformer model is a deep learning model based on a self-attention mechanism, primarily used to process sequential data such as text and speech. The Temperature parameter is a key hyperparameter in the Transformer model used to control the randomness and creativity in the text generation process.

[0089] Step S222: Perform fission processing on the initial prompt words based on role obfuscation fission, translation obfuscation strategy and logic escape algorithm to obtain fission prompt words.

[0090] The intent information included in the fission prompt is the same as that of the corresponding initial prompt.

[0091] Perform prompt word fission generation. This step employs 79 adversarial fission algorithms, whose unique strategies include:

[0092] Role confusion and fission: Force models to play specific professions or authority roles.

[0093] Translation obfuscation strategy: Translate the intended meaning into a non-mainstream language and embed it in the code segment.

[0094] Logical escape algorithm: Constructs nested logical prompts, allowing the model to ignore initial safety instructions when processing logical chains. Nested logical prompts refer to building multi-layered nested instructions within the prompt word using logical structures such as conditions, parallelism, and negation, to guide large models to more accurately understand complex tasks or generate content that conforms to multiple constraints.

[0095] Multilingual obfuscation algorithm: malicious requests are pieced together in fragments across languages ​​(such as Chinese / English / French / Japanese).

[0096] Role-Play Jailbreak Algorithm: This algorithm utilizes the Role-Play Jailbreak strategy to hide instructions within a specific script of a fictional character. Role-Play Jailbreak is a technique that bypasses security restrictions by allowing a large language model to enter a specific role, enabling the model to generate content that would be prohibited under normal circumstances.

[0097] Command-level evasion: Hides the attack payload in code comments or technical documentation format to induce the model to execute.

[0098] Step S230: Construct a preset prompt word library based on the initial prompt word, similar prompt words, and fission prompt words.

[0099] Each prompt word in the preset prompt word library includes intent information.

[0100] Specifically, a pre-defined prompt word library is constructed based on the initial prompt words, similar prompt words, and fission prompt words. K-means clustering or a legal and regulatory dictionary mapping algorithm is employed. The generated millions of prompt words are structurally categorized according to legal regulations and business security dimensions, establishing a hierarchical management system of "cluster-intent-variant" to facilitate rapid extraction of prompt words later. K-means clustering is a common unsupervised learning algorithm used to divide a dataset into K clusters. The goal of this algorithm is to assign data points to K clusters such that the distance between each data point and the centroid of its cluster is minimized, thereby achieving rapid clustering of prompt words based on the same intent.

[0101] Beneficial effects: By supporting various advanced fission strategies such as semantic preservation, expression obfuscation fission, cognitive path shift, and role obfuscation, it is possible to simulate the attack thinking of real hackers and generate a dynamically mutated prompt word library.

[0102] In some embodiments, such as Figure 5 As shown, step S120 includes:

[0103] Step S121: Based on the test requirements of the target test mode, extract prompt words from the preset prompt word library as target prompt words.

[0104] Specifically, multiple prompt words are extracted, each serving as a target prompt word. Different target testing modes have different testing requirements. The batch prompt word testing mode requires a large number of prompt words with different intentions to be tested in parallel. The red team adversarial self-evolution mode requires prompt words to be modified and evolved. The multi-turn contextual guidance mode requires prompt words with progressive relationships.

[0105] Step S122: Use the intent information included in the target prompt as the test intent of the test input content, and configure the scenario for the target prompt to construct the test input content including the test intent and test background.

[0106] Specifically, a scenario model for a specific domain is loaded, and background knowledge is injected into the prompts. This maps general attack intent to specific business scenarios.

[0107] Step S123: Input the test input content into the application model under test to test the application model under test.

[0108] Specifically, the test input is input into the application model under test, and the application model under test is then tested.

[0109] Beneficial effects: The steps for testing the application model under test by constructing test input content based on a preset prompt word library in the target test mode are further refined, enriching the embodiments of the present invention. At the same time, by configuring the prompt words in a scenario, general attack intent can be mapped to a specific business scenario.

[0110] In some embodiments, when performing model testing, the evaluation is triggered by a distributed scheduler, i.e., triggering background multi-threaded asynchronous concurrent test tasks. By employing a collaborative architecture of AsyncIO and ThreadPoolExecutor, the high-concurrency I / O blocking problem of model calls is solved. In Python concurrent programming, AsyncIO (asynchronous I / O) and ThreadPoolExecutor (thread pool) are two complementary concurrency models. Using the ThreadPoolExecutor architecture, combined with a database logical lock retry mechanism, ensures rapid recovery when large-scale concurrent requests cause database contention. This allows for parallel testing of multiple models simultaneously, or parallel testing of multiple models.

[0111] Beneficial effects: The combination of concurrent scheduling mechanism and lightweight model enables tens of thousands of adversarial probing tests per hour to be supported in a single machine environment, which is more than 100 times more efficient than manual testing, and does not require expensive A100 / H100 computing power clusters.

[0112] In some embodiments, such as Figure 6 As shown, step S130 includes:

[0113] Step S131: Obtain the test output of the application model under test response test.

[0114] Specifically, the output response of the model under test is captured in real time using the WebSocket streaming protocol, and the response is cleaned. The WebSocket streaming protocol is a TCP-based network communication protocol (standard RFC 6455).

[0115] Step S132: Based on a preset algorithm, extract the model thinking process content from the test output content to obtain the processed test output content.

[0116] Specifically, the preset algorithm is the strip_thinking algorithm (a data processing technique in the development of large language model (LLM) applications), which uses the strip_thinking algorithm to strip away the internal thinking process of the model and extract the pure response text.

[0117] Step S133: Based on the 3D label mapping algorithm, dimension labeling is performed on the processed test output content to obtain the target test output content including the labeled dimensions.

[0118] Specifically, a 3D label mapping algorithm is used to automatically label each returned result from three dimensions: "Intent", "Scene", and "Strategy".

[0119] Beneficial effects: The adjudication system abandons simple binary judgments and constructs a three-dimensional modeling space based on "Intent - Scene - Strategy". It can accurately distinguish whether the model is "performing a malicious attack" or "aligning with a compliance stance".

[0120] In some embodiments, when the target testing mode is a red team adversarial self-evolution mode or a multi-round context-induced mode; after step S140, the method further includes:

[0121] If the risk level of the application model under test is determined to be risk-free, then modify the test input and test the application model again. Repeat this step iteratively until the risk level of the application model under test is determined to be risky, or the number of iterations reaches the preset number.

[0122] Specifically, when the target test mode is the red team adversarial self-evolution mode, the attack model performs real-time semantic correction based on the "rejection logic" given by the tested model, and iteratively repeats the above steps S120-S140 until the risk level of the tested application model is determined to be risky, or the number of iterations reaches the preset number, thereby generating the most penetrating variant prompt words through multiple generations of evolution.

[0123] When the target testing mode is a multi-turn context-induced mode, the logic of this mode adopts a step-by-step induction algorithm. A preset dialogue step length (i.e., a preset number of times) is used. Trust is first established through several rounds of safe background, and then logical traps are gradually introduced to uncover hidden weaknesses in the model's long context. Each round of dialogue is a test, and the test input needs to be modified before the next round (i.e., gradually induced according to the preset logic). Therefore, steps S120-S140 are iteratively repeated until the risk level of the tested application model is determined to be risky, or the preset number of iterations (i.e., the dialogue step length) is reached.

[0124] Beneficial effects: When the target test mode is a red team adversarial self-evolutionary mode or a multi-round context-induced mode, if a single test is risk-free, it is necessary to modify the test input and perform iterative repeated testing until the risk level of the application model under test is determined to be risky, or the number of iterations reaches a preset number. This improves the testing scheme for the red team adversarial self-evolutionary mode or the multi-round context-induced mode, thereby improving the solution of this invention.

[0125] Simultaneously, by leveraging the game between the "attack model" and the "target model," dynamic mutations are performed based on the rejection reasons (refusal analysis) of the target model. Semantic analysis of failed cases is conducted using a large model to accurately identify defense boundaries and generate more targeted variant prompts accordingly.

[0126] In some embodiments, to address the common "false positive" problem in traditional security assessments (i.e., the model outputs garbled text or irrelevant content and is mistakenly judged as bypassed), an independent auditing step is introduced. Specifically:

[0127] Test input includes the test intent, such as... Figure 7 As shown, after step S140, the following steps are also included:

[0128] Step S310: If the risk level of the application model under test is determined to be risky, then check whether the intent dimension of the target test output content matches the test intent.

[0129] A second round of expert auditing is conducted by a local testing center. The Auditor-Net architecture (a deep neural network architecture designed for network security auditing scenarios) is employed. For use cases where risks have been identified, an independent audit model performs "reverse engineering" to verify whether the response truly achieved the attacker's original intent.

[0130] Step S320: If a match is found, the current test is determined to be a valid test, and the risk level is output.

[0131] If a match is found, it means that the response truly achieved the attacker's original intent, the current test is a valid test, and the risk level is output.

[0132] Step S330: If there is no match, the current test is determined to be invalid, and the application model under test is retested.

[0133] If there is no match, it means that the response did not achieve the attacker's original intent, and the current test is invalid. In this case, the application model under test needs to be tested again.

[0134] Beneficial Effects: This step employs a Fulfillment Auditor to verify whether the model's response truly meets the malicious target's payload requirements. After risk assessment, the "Fulfillment Auditor" provides a secondary confirmation of the achievement of the original intent and the actual response, ensuring that every case marked as "risk" represents a substantial security vulnerability breach. This significantly reduces evaluation bias caused by model "misbehavior" and enhances the authority of the test results. Compared to traditional testing systems, this embodiment greatly reduces the false positive rate of test results.

[0135] A distributed multi-agent architecture was constructed, consisting of an "Attacker," a "Judge," and an "Auditor." Each test task flows among these three agents, forming a tight game chain. The Auditor performs logical consistency checks on the Judge's reasoning. If the Judge determines a task as "risky" but cannot provide corresponding three-dimensional label evidence, the system automatically triggers a reassessment, thus ensuring the objectivity of the evaluation at the algorithm level.

[0136] In some embodiments, the test input includes test intent, test background, and test strategy; the target test mode is a red team adversarial self-evolution mode; such as Figure 8 As shown, after step S320, the following steps are also included:

[0137] Step S410: Obtain the test intent, test background, and test strategy of the test input content.

[0138] The testing intent refers to the intended information of the prompt, while the testing context is the scenario in which the prompt is configured. The strategy defines whether the prompt is harmful or harmless; harmful prompts are those with direct malicious intent, while harmless prompts are those without malicious intent.

[0139] Step S420: Construct an attack strategy based on the test intent, test background, and test strategy.

[0140] The attack strategy includes prompts and attack background.

[0141] Construct an attack strategy that includes cue words and attack context based on the test intent, test background, and test strategy. The attack context is the test background, and the cue words are prompts that include the test intent.

[0142] Step S430: Store the attack strategy in the preset attack library in the Red Team's self-evolution mode.

[0143] The preset attack library contains a repository of attack strategies for the preferred attack methods when conducting subsequent tests based on the red team adversarial self-evolution mode.

[0144] By initiating attack weapon effectiveness tracking and real-time statistics on the success rate of each fission strategy, once a high-risk bypass path is detected, successful tactics are automatically and in real-time fed back to the red team's self-evolution mode for strategy weighting, forming a complete self-healing and evolutionary closed loop.

[0145] In some embodiments, the method further includes: aggregating the assessment data to generate structured analysis results covering risk heatmaps, model robustness curves, and sensitive leakage trends.

[0146] In some embodiments, a backend template engine can generate a deep test report in Docx format with a single click, providing targeted model alignment and defense hardening suggestions. It also offers the export of a comprehensive report and download of governance recommendations, completing the entire evaluation process from "risk discovery" to "remediation and reinforcement."

[0147] Through the above implementation methods, automatic expansion of prompt words, joint analysis of multiple models, and automated execution of the evaluation process can be achieved, thereby significantly improving the efficiency and accuracy of large model evaluation.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A fuzz testing method for multimodal large model applications, characterized in that, The method includes: Based on the preset testing requirements, select the corresponding testing mode as the target testing mode; wherein, the target testing mode includes batch prompt word testing mode, red team adversarial self-evolution mode and multi-round contextual guidance mode; In the target test mode, test input content is constructed based on a preset prompt word library to test the application model under test; The process involves: acquiring the test output content of the application model under test responding to the test; performing dimension labeling processing on the test output content based on a 3D label mapping algorithm to obtain target test output content including labeled dimensions; wherein, the labeled dimensions include intent dimension, scenario dimension, and strategy dimension; and including: acquiring the test output content of the application model under test responding to the test; extracting the model thinking process content from the test output content based on a preset algorithm to obtain processed test output content; and performing dimension labeling on the processed test output content based on a 3D label mapping algorithm to obtain target test output content including labeled dimensions. The risk level of the tested application model is determined based on the labeling dimensions of the target test output content; wherein the risk level includes no risk and risky.

2. The method according to claim 1, characterized in that, Before the step of constructing test input content based on a preset prompt word library and testing the application model under test in the target test mode, the method further includes: Based on the preset test requirements, the environment parameters of the application model under test are configured; wherein, the environment parameters include API endpoints, headers, and cookie streams.

3. The method according to claim 1, characterized in that, Before the step of constructing test input content based on a preset prompt word library and testing the application model under test in the target test mode, the method further includes: Based on preset test requirements, initial prompt words are established; wherein, the initial prompt words include intent information; The initial prompt words are subjected to similarity generation and fission generation processes to obtain similar prompt words and fission prompt words; The preset prompt word library is constructed based on the initial prompt word, the similar prompt words, and the fission prompt words; wherein, each prompt word in the preset prompt word library includes intent information.

4. The method according to claim 3, characterized in that, The step of establishing initial prompt words based on preset test requirements includes: Establish starting prompts based on pre-defined testing requirements; The starting prompt word is structurally labeled based on N-gram feature extraction and clustering algorithms, and the structured-labeled starting prompt word is stored as the initial prompt word; wherein, the structured label includes attack method level, legal violation category, and business risk area, and the structured label is used to construct the intent information.

5. The method according to claim 3, characterized in that, The step of performing similarity generation and fission generation processing on the initial prompt words to obtain similar prompt words and fission prompt words includes: A semantic restatement algorithm is used to generate similar prompts from the initial prompts; wherein the intent information included in the similar prompts is the same as the intent information of the corresponding initial prompts. The initial prompt word is split based on role obfuscation splitting, translation obfuscation strategy and logic escape algorithm to obtain the split prompt word; wherein the intent information included in the split prompt word is the same as the intent information of the corresponding initial prompt word.

6. The method according to claim 3, characterized in that, The step of constructing test input content based on a preset prompt word library and testing the application model under test in the target test mode includes: Based on the testing requirements of the target testing mode, prompt words are extracted from the preset prompt word library as target prompt words; The intent information included in the target prompt words is used as the test intent of the test input content, and the target prompt words are configured with a scenario to construct test input content that includes test intent and test background; The test input content is input into the application model under test to test the application model under test.

7. The method according to claim 1, characterized in that, When the target testing mode is either the red team adversarial self-evolution mode or the multi-round context-induced mode; After determining the risk level of the application model under test based on the labeled dimensions of the target test output content, the following steps are included: If the risk level of the application model under test is determined to be risk-free, then the test input content is modified, and the application model under test is tested again. This step is repeated iteratively until the risk level of the application model under test is determined to be risky, or the number of iterations is repeated to a preset number.

8. The method according to claim 1, characterized in that, The test input includes the test intent; After the step of determining the risk level of the application model under test based on the labeled dimensions of the target test output content, the method further includes: If the risk level of the application model under test is determined to be risky, then it is checked whether the intent dimension of the target test output content matches the test intent. If a match is found, the current test is determined to be a valid test, and the risk level is output. If there is a mismatch, the current test is determined to be invalid, and the application model under test is retested.

9. The method according to claim 8, characterized in that, The test input also includes the test background and test strategy; the target test mode is the red team adversarial self-evolution mode. After the step of determining the current test as a valid test and outputting the risk level if a match is found, the method further includes: Obtain the test intent, test background, and test strategy from the test input content; An attack strategy is constructed based on the test intent, the test background, and the test strategy; wherein, the attack strategy includes prompts and attack background. The attack strategy is stored in the preset attack library in the red team confrontation self-evolution mode.

Images