Industrial control protocol intelligent fuzzy test method and system based on multi-agent large model

Abstract

The invention discloses an industrial control protocol intelligent fuzzy test method and system based on a multi-agent large model, and relates to an industrial information security test method and system.According to the industrial control protocol intelligent fuzzy test method and system based on the multi-agent large model, the large model is enhanced by integrating the field of the retrieval enhancement generation (RAG) and the low-rank quantization adaptation (QLoRA) technology, and a multi-agent cooperation mechanism is combined; and full-automatic, high-precision and end-to-end fuzzy testing of the industrial protocol is realized. The system generates seed data meeting protocol specifications through the seed generation agent, the test case generation agent executes field, structure and semantic variation based on the seed data, efficient test cases are dynamically generated, a test strategy is optimized in real time through the feedback analysis and strategy adjustment agent, and the vulnerability discovery capability is improved. By dynamically adjusting the variation strategy and optimizing the generation of the test case, the coverage, the accuracy and the efficiency of the test are improved, and the security and the vulnerability detection capability of the industrial control system are remarkably improved.

Description

Technical Field

[0001] This invention pertains to industrial information security testing methods and systems, specifically relating to an intelligent fuzzy testing method and system for industrial control protocols based on a multi-agent large model. Background Technology

[0002] With the deep integration and widespread application of information and communication technologies in the industrial control field, traditional industrial control architectures are gradually transforming into new industrial control architectures based on advanced network networking. To ensure real-time and reliable interconnection of industrial production elements, the original strictly hierarchical network system has been broken down, forming a new flattened network architecture. All key production elements in the industrial field are fully connected to achieve cross-level data transmission. Programmable Logic Controllers (PLCs) have become the bridge for data communication, and their data carriers, Industrial Control Protocols (ICPs), are also transforming into transparent and open forms. From a protocol standard perspective, they form layered decoupling and interconnection, allowing upper-layer protocols to serve as payloads for any lower-layer protocol, as payloads in wireless networks, and even evolve independently. However, ICPs were initially designed primarily for functionality and efficiency, with insufficient consideration for security, making them a primary target for network attacks. In recent years, frequent attacks targeting industrial protocols have exposed serious security deficiencies in ICPs, highlighting the importance of comprehensive vulnerability discovery for industrial protocols.

[0003] Traditional fuzzing is an effective vulnerability discovery method that induces anomalies in the target system by generating a large number of random or pseudo-random inputs. However, most traditional fuzzing methods and systems are geared towards network and text protocols, which typically have flexible structures and low stability requirements. In contrast, highly structured industrial protocols usually have fixed fields and strict format constraints. Any slight deviation can cause the target system to reject the input, which poses a significant challenge to test case generation. On the other hand, the operation of industrial control systems directly affects physical equipment and safe production. Protocol design usually focuses on stability and fault tolerance, and different manufacturers and equipment implement highly customized protocol versions. Existing methods and systems are difficult to adapt to and extend to multi-protocol environments, further increasing the complexity of industrial control protocol vulnerability discovery.

[0004] In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities in semantic understanding, generation, and contextual reasoning, providing novel approaches to fuzz testing of complex protocols. However, existing LLM applications mostly focus on network or text protocols, typically serving as extensions of existing tools, and their effectiveness in handling the context of structured protocols is limited. Furthermore, methods like ChatAFL generally rely on manually defined rules, data cleaning, and manual adjustments, making it difficult to meet the demands of efficient automated testing for industrial protocols. While cloud-based training using the latest OpenAI model APIs can achieve good testing performance, the act of uploading highly confidential industrial traffic to cloud servers for processing inherently poses significant security risks. Additionally, cloud-to-end computation limits network throughput, severely impacting testing efficiency.

[0005] In summary, in order to solve the problems of difficulty in adapting to complex protocol environments in traditional industrial control protocol fuzzing and the heavy reliance on manual adjustment and tool calls by methods based on large language models, there is an urgent need for a high-performance fuzzing method and system that can automatically adapt to the complexity of industrial protocols and meet the requirements of fully automated end-to-end testing. Summary of the Invention

[0006] This invention proposes an intelligent fuzz testing method and system for industrial control protocols based on a multi-agent large model, aiming to overcome the limitations of traditional fuzz testing in industrial protocol environments. The method and system achieve fully automated, end-to-end, high-precision fuzz testing of industrial protocols by integrating Retrieval Enhanced Generation (RAG), Low-Rank Quantization Adaptation (QLoRA) domain-enhanced large models, and a multi-agent collaboration mechanism. It can dynamically generate high-quality test cases and uncover potential security vulnerabilities.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A method for intelligent fuzz testing of industrial control protocols based on a multi-agent large model, the method comprising the following steps:

[0009] Building a Domain-Enhanced Large Model: By integrating Retrieval Enhanced Generation (RAG) and Low-Rank Quantization Adaptation (QLoRA) techniques, a large language model is built that can parse industry protocol constraints, optimize test case generation, and perform adaptive feedback adjustments.

[0010] Seed generation agent: Based on industry protocol specifications, it uses RAG technology to dynamically retrieve protocol constraints and generates seed data that conforms to the protocol specifications through multi-agent collaboration;

[0011] Test case generation agent: Utilizing seed data and combined with a large language model fine-tuned by QLoRA, it performs field mutation, structural mutation, and semantic mutation to generate diverse protocol test cases;

[0012] Feedback Analysis and Strategy Adjustment Agent: Analyzes the response data of the target system in real time, and adjusts the testing strategy through cluster analysis and anomaly scoring to optimize the coverage of test cases and vulnerability discovery capabilities;

[0013] Communication and interaction module: intercepts real-time data, metadata and payload of industrial control protocols, and ensures the efficiency, stability and scalability of fuzz testing through real-time communication and task synchronization between multiple agents.

[0014] The aforementioned intelligent fuzz testing method for industrial control protocols based on a multi-agent large model includes:

[0015] The RAG module is used to dynamically extract protocol constraint information from knowledge bases such as industry protocol specifications and vulnerability databases (CVEs), and to provide contextual guidance during seed generation and test case generation.

[0016] The QLoRA fine-tuning module is used to optimize large language models in resource-constrained environments. It improves the adaptability of protocol tasks by fine-tuning the model weights through a low-rank matrix.

[0017] The aforementioned intelligent fuzz testing method for industrial control protocols based on a multi-agent large model includes the following functional sub-modules for seed generation agents:

[0018] The protocol knowledge dynamic retrieval submodule dynamically retrieves relevant rules and constraints from the industry protocol knowledge base via RAG;

[0019] The field extraction and verification submodule uses chained reasoning (CoT) to parse protocol data packet fields step by step, ensuring that the parsing logic is clear and the results are reliable.

[0020] The dynamic template generation submodule automatically generates test seed templates based on the field extraction results and the protocol rules retrieved by RAG.

[0021] The aforementioned intelligent fuzz testing method for industrial control protocols based on a multi-agent large model includes the following functional sub-modules for generating test cases:

[0022] The input acquisition submodule obtains structured seed data from the seed generation agent and obtains test strategy priority and mutation intensity adjustment parameters from the feedback analysis agent.

[0023] The mutation logic submodule performs field mutations, structural mutations, and semantic mutations based on the input data to generate diverse test cases;

[0024] The output generation submodule formats the mutated test cases into JSON format for subsequent fuzz testing.

[0025] The aforementioned intelligent fuzzy testing method for industrial control protocols based on a multi-agent large model includes the following functional sub-modules for feedback analysis and strategy adjustment agents:

[0026] The feedback data acquisition and classification submodule collects response data from the system under test and classifies the feedback into normal response, abnormal response and critical anomaly based on predefined criteria.

[0027] The abnormal behavior analysis submodule uses a weighted scoring system to quantify anomalies and extracts potential vulnerability patterns from historical test data.

[0028] The dynamic strategy adjustment submodule dynamically adjusts mutation density and direction based on feedback analysis results to improve test coverage and attack power.

[0029] The intelligent fuzz testing method for industrial control protocols based on a multi-agent large model, wherein the communication interaction module includes the following functional sub-modules:

[0030] The real-time traffic capture submodule intercepts real-time data packets of the target protocol and extracts protocol metadata and payload.

[0031] The test case injection submodule enables real-time injection and response monitoring of test cases, ensuring that the injected test cases can be seamlessly integrated into the protocol communication link.

[0032] The multi-agent coordination submodule, based on task queues and asynchronous message passing mechanisms, enables efficient data exchange and task collaboration among seed generation agents, test case generation agents, and feedback analysis agents.

[0033] A smart fuzzy testing system for industrial control protocols based on a multi-agent large model, the system comprising:

[0034] The domain-enhanced large model module is used to build large language models through RAG and QLoRA technologies. It has industrial protocol awareness capabilities and can realize the parsing and test case generation of various industrial protocols.

[0035] The seed generation agent module generates seed data that conforms to the protocol specifications by using the protocol knowledge base and context information;

[0036] The test case generation proxy module generates efficient test cases based on seed data and feedback analysis results through field, structure, and semantic variations.

[0037] The feedback analysis and strategy adjustment agent module analyzes the target system's response in real time and adjusts the testing strategy based on the feedback to improve vulnerability discovery capabilities.

[0038] The communication and interaction module supports task synchronization and test case injection between various agent modules, ensuring efficient and stable testing processes.

[0039] The aforementioned intelligent fuzz testing system for industrial control protocols based on a multi-agent large model includes a domain-enhanced large model comprising a RAG module and a QLoRA fine-tuning module. The RAG module is used to dynamically extract protocol constraints and industrial control vulnerability database information, while the QLoRA module reduces computational resource requirements by freezing the basic parameters of the LLM and training the inserted low-rank matrix, thereby enabling the large model to adapt to low-resource environments.

[0040] The aforementioned intelligent fuzz testing system for industrial control protocols based on a multi-agent large model comprises a seed generation agent including a protocol knowledge dynamic retrieval submodule, a field extraction and verification submodule, and a dynamic template generation submodule. Combined with RAG technology, it dynamically retrieves relevant rules and constraints from protocol specification documents, CVE vulnerability databases, and real traffic logs, providing contextual information for generating seed data that conforms to the protocol specification. The test case generation agent includes an input acquisition submodule, a mutation logic submodule, and an output generation submodule. Combined with CoT technology, it generates diverse and protocol-compliant test cases, ensuring comprehensive discovery of protocol vulnerabilities.

[0041] The aforementioned intelligent fuzz testing system for industrial control protocols based on a multi-agent large model includes a feedback analysis and strategy adjustment agent comprising a feedback data acquisition and classification submodule, an abnormal behavior analysis submodule, and a dynamic strategy adjustment submodule. It quantifies the severity of abnormal responses through a weighted scoring mechanism and adjusts testing strategies based on the scoring results to ensure maximum test coverage and vulnerability discovery capabilities. The communication interaction module includes a real-time traffic capture submodule, a test case injection submodule, and a multi-agent coordination submodule. It intercepts and processes real-time data of the target protocol using the traffic capture tool Scapy, and further employs the ZeroMQ message passing mechanism. Through task queues and asynchronous message passing, it achieves real-time communication, task synchronization, and test case injection between multiple agents, ensuring an efficient testing process and system stability.

[0042] Meanwhile, this invention adopts a modular multi-agent design and consists of the following core components, including:

[0043] Domain-enhanced large model, seed generation agent, test case generation agent, feedback analysis and strategy adjustment agent, communication and interaction module; among which:

[0044] The aforementioned domain-enhanced large model, based on RAG and QLORA fine-tuning technologies, enables the LLM to have industrial protocol awareness capabilities, realize protocol constraint parsing, test case generation optimization, and feedback adaptive adjustment, and is suitable for various industrial protocol environments, such as mainstream industrial control protocols such as Modbus / TCP, S7Comm, and Ethernet / IP.

[0045] The seed generation agent generates seed data that conforms to the protocol specifications based on industry protocol specifications and a contextual knowledge base. It dynamically retrieves protocol constraints (such as function code range and field dependencies) through RAG and combines this with LLM to generate protocol-aware input seeds, ensuring their legitimacy and diversity.

[0046] A test case generation agent utilizes seed data to generate protocol-valid test cases, including field mutations, structural mutations, and semantic mutations. QLoRA is employed to fine-tune the LLM task, achieving efficient field mutation and semantic reasoning. Feedback-driven strategy adjustments dynamically optimize the mutation range and density, ensuring high efficiency and vulnerability discovery capabilities.

[0047] The feedback analysis and strategy adjustment agent analyzes the target system's response (such as error codes, timeouts, and abnormal behavior) in real time, quantifies the severity of anomalies, and adjusts the testing strategy for the next fuzzing cycle. Through clustering and weight analysis of runtime feedback, it identifies key fields and high-risk mutation directions, improving testing efficiency and coverage.

[0048] The communication and interaction module is responsible for real-time communication between multiple agents, task synchronization, and test case injection into the target system. It employs an asynchronous communication mechanism to support high concurrency and low-latency data interaction, ensuring the efficiency and stability of the testing process.

[0049] Preferably, the domain-enhanced large model includes:

[0050] RAG is used to dynamically extract protocol constraint information from knowledge bases such as industry protocol specifications and vulnerability databases (CVEs), and to provide contextual guidance during seed generation and test case generation.

[0051] QLoRA fine-tuning is used to optimize large language models in resource-constrained environments. It improves the adaptability of industrial protocol fuzzing tasks by fine-tuning model weights through low-rank matrices.

[0052] Preferably, the RAG adopts a "retrieval-reading" mode to provide the LLM with task-related knowledge context, thereby enhancing its protocol awareness capabilities. Traditional LLMs have limited context windows, making it difficult to directly embed complete protocol specifications. However, this invention, through extensive collection of protocol documents, vulnerability description libraries, and real industrial traffic for content extraction and context construction, enables the LLM to invoke protocol knowledge in real time, improving the effectiveness of test cases.

[0053] Preferably, the protocol document submodule in the RAG is used to parse industrial protocol documents and extract hierarchical index information, including command structure, field format, data constraints, function code definitions, and error responses. This submodule automatically constructs a protocol knowledge base through text parsing and rule matching technologies for reference in test case generation.

[0054] The vulnerability description library submodule integrates databases of known industrial control protocol vulnerabilities (such as CVE, CNVD, and CNNVD) and extracts known attack vectors such as protocol field anomalies, function abuse, and buffer overflows from the vulnerability descriptions. It employs an automatic association and matching mechanism to associate vulnerability information with protocol field constraints, providing vulnerability-oriented contextual information for test case generation, thereby achieving precise fuzz testing.

[0055] The Real Industrial Traffic Analysis submodule collects communication traffic from real industrial network environments, processes it into hexadecimal format, and enriches the protocol knowledge base with unannotated text to enhance the real-world applicability of test cases.

[0056] The context fusion submodule dynamically retrieves protocol-related field information during test task execution, matches key terms (such as "function code" and "data field"), recursively parses relevant chapters, and builds an industrial protocol knowledge base. Combined with task instructions, it generates background enhancement prompts and adopts a task adaptive query method to dynamically adjust the weight of the retrieved content, so that the generated test cases not only conform to protocol specifications but also have efficient vulnerability discovery capabilities.

[0057] Preferably, the QLoRA fine-tuning achieves low-cost, high-performance protocol adaptation by freezing the basic parameters of the LLM and training only the inserted low-rank matrix. Traditional LLM fine-tuning requires large-scale datasets and high computational resources, while QLoRA uses 4-bit quantization to reduce memory requirements and ensures that the model adapts to the specific tasks of industrial protocols through low-rank adjustment.

[0058] The QLoRA module includes:

[0059] The model quantization submodule is used to quantize LLM weights to 4-bit to reduce computational resource consumption and make it suitable for embedded or low-resource industrial environments.

[0060] The low-rank adaptation submodule ensures the adaptability of the protocol task by inserting a low-rank matrix, while maintaining the overall stability of the model, enabling specific layers of the LLM to adapt to industrial protocol tasks.

[0061] Preferably, the RAG module works in conjunction with the QLoRA module, enabling the domain-enhanced LLM to efficiently execute protocol awareness tasks in low-resource environments, and improving the relevance of seed and test case generation through task customization fine-tuning.

[0062] Preferably, the seed generation agent in the industrial control protocol fuzzing method and system is used to parse real-time protocol traffic and generate high-quality test seeds that conform to protocol rules. The agent automatically transforms the original protocol data into structured seed data through real-time traffic capture, dynamic retrieval of protocol knowledge, and LLMs-driven chain reasoning technology, thereby improving the efficiency and accuracy of fuzzing. It is applicable to various industrial protocols such as Modbus / TCP, S7Comm, and Ethernet / IP.

[0063] Preferably, the seed generation agent includes the following functional sub-modules:

[0064] The protocol knowledge dynamic retrieval submodule uses RAG to dynamically retrieve relevant rules and constraints from the industry protocol knowledge base, including protocol specification documents, CVE vulnerability databases and real traffic logs, to provide contextual information support for protocol parsing;

[0065] The field extraction and verification submodule uses chained reasoning to progressively parse protocol data packet fields, ensuring clear parsing logic and reliable results. Parsing includes protocol type identification, key field extraction, and field value verification to ensure fields conform to protocol constraints.

[0066] The dynamic template generation submodule automatically generates test seed templates based on the field extraction results and the protocol rules retrieved by RAG; it converts the parsing and template generation results into structured data format (JSON) for use by the subsequent fuzzing module.

[0067] Preferably, the test case generation agent includes the following functional sub-modules:

[0068] The input acquisition submodule obtains structured seed data from the seed generation agent and test strategy priority and mutation strength adjustment parameters from the feedback analysis agent. The test strategy defines the priority and mutation strength of mutation types to ensure that the generated test cases meet the current test requirements.

[0069] The mutation logic submodule performs field mutations, structural mutations, and semantic mutations based on the input data to generate diverse test cases;

[0070] The dynamic strategy adjustment module dynamically adjusts the mutation intensity based on the anomaly score provided by the agent, prioritizes testing high-risk fields, and adjusts the mutation strategy in real time, thereby improving the coverage of critical vulnerabilities in the target system.

[0071] The output generation submodule formats the mutated test cases into JSON and hexadecimal formats, ensuring that they can be directly used for injection testing of the target system; the generated test cases are verified to have protocol compliance and high attack power, making them suitable for vulnerability discovery.

[0072] Preferably, the feedback analysis and strategy adjustment agent includes the following functional sub-modules:

[0073] The feedback data acquisition and classification submodule collects response data from the system under test (SUT) and classifies the feedback into normal responses, such as protocol-compliant confirmation messages, abnormal responses, such as unexpected error codes and delays, and critical anomalies, such as timeouts, system crashes, or resource overload.

[0074] The abnormal behavior analysis submodule uses a weighted scoring system to quantify detected anomalies, assess their severity, extract potential vulnerability patterns from historical test data, accurately quantify the impact of anomalies, and prioritize the location of potential vulnerability areas to improve testing efficiency.

[0075] The dynamic strategy adjustment submodule dynamically adjusts the mutation density and direction based on the results of feedback analysis, prioritizing testing of high-risk fields (such as error function codes) to improve test case coverage and attack power.

[0076] Preferably, the communication interaction module includes the following functional sub-modules:

[0077] The real-time traffic capture submodule is responsible for capturing real-time data packets of the target protocol using the traffic capture tool Scapy, extracting protocol metadata and payload, and providing input data for subsequent parsing and seed generation;

[0078] The test case injection submodule dynamically injects generated test cases into the ongoing communication stream, while capturing the SUT's response data for feedback analysis. It supports real-time injection and response monitoring of mutation test cases, ensuring that the injected test cases can be seamlessly integrated into the protocol communication link.

[0079] The multi-agent coordination submodule, based on task queues and asynchronous message passing mechanisms, enables workflow coordination between seed generation agents, test case generation agents, and feedback analysis agents. It adopts ZeroMQ's PUB / SUB model, supports real-time data distribution and many-to-many communication, and ensures task synchronization and data consistency among agents.

[0080] The technical solution of the present invention has the following beneficial effects:

[0081] 1. This invention, through a multi-agent architecture design combining RAG and QLoRA, automates the entire process from test seed generation and test case mutation to feedback analysis and strategy adjustment. Compared to traditional fuzz testing methods that rely on manual intervention, this invention significantly reduces operational complexity and improves testing efficiency and accuracy.

[0082] 2. Utilizing QLoRA technology for 4-bit quantization of large language models significantly reduces memory requirements, enabling the framework to run efficiently on industrial-grade GPU servers or resource-constrained environments. Simultaneously, prompt word engineering and context optimization mechanisms ensure the model's performance advantages in complex protocol tests.

[0083] 3. This invention adopts a modular design, supports multiple mainstream industrial protocols (such as Modbus / TCP, S7Comm, Ethernet / IP), and can be quickly extended to new protocols (such as OPC UA) or custom testing strategies. Meanwhile, the loosely coupled architecture of the communication interaction module and standardized data interfaces (such as JSON) support dynamic loading of protocol parsers and mutation modules, meeting the needs of complex testing scenarios without large-scale framework adjustments.

[0084] In summary, the technical solution of this invention overcomes the shortcomings of existing industrial protocol fuzzing methods and systems in terms of intelligence, automation, test coverage, and scalability, significantly improving the efficiency and depth of industrial protocol vulnerability discovery, and has broad application value and promotion prospects. Attached Figure Description

[0085] Figure 1 This is a schematic diagram of the intelligent fuzzy testing method for industrial control protocols based on a multi-agent large model in Example 1;

[0086] Figure 2 This is a schematic diagram of the large-scale industrial control protocol domain enhancement model based on RAG and QLORA fine-tuning technologies in Example 1;

[0087] Figure 3 This is a diagram of the intelligent fuzzy testing system architecture for industrial control protocols based on a multi-agent large model in Example 2.

[0088] Figure 4 This is a schematic diagram of the multidimensional evaluation method for fuzzy testing of industrial control protocols in Example 3;

[0089] Figure 5 This refers to several industrial control protocol vulnerability information autonomously discovered by the multi-agent large model fuzzy testing system described in Example 3. Specific implementation methods

[0090] To make the objectives, content, and advantages of this invention clearer, the specific embodiments of this invention will be described in further detail below with reference to the accompanying drawings and examples. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to help understand the content of this application, and should not be construed as limiting this application. Other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are all within the scope of protection of this invention.

[0091] Example 1

[0092] This embodiment discloses an intelligent fuzz testing method and system for industrial control protocols based on a multi-agent large model. It can be executed by computer devices or virtual machines with 12GB or more of video memory, in order to reduce the impact of hardware on fuzz testing efficiency.

[0093] Preferably, the intelligent fuzz testing method and system flow for industrial control protocols based on a multi-agent large model provided in this embodiment can be found in the following reference: Figure 1 It may include, but is not limited to, the following steps:

[0094] S1. Constructing a domain-enhanced large model: By combining Retrieval Enhancement Generation (RAG) and Low-Rank Quantization Adaptation (QLoRA) techniques, efficient adaptation and optimization of the large language model in industrial protocol fuzz testing are achieved. This model is used for protocol awareness, seed generation, mutation test case construction, and feedback analysis. Please refer to the schematic diagram. Figure 2 Specifically, in this embodiment, the implementation process of the domain enhancement big model is as follows:

[0095] S101. Protocol Document Library Parsing: Collects protocol documents from industrial control system (ICS) related protocol standards (such as Modbus / TCP, S7Comm, Ethernet / IP, etc.) and converts them into a unified parsable format;

[0096] Regular expressions are used to match key terms in the text, such as "function code," "command structure," and "error response." The chapter structure of the protocol document is analyzed, and a hierarchical index is generated based on the title format, including chapter number, title, and page range. Constraint information of protocol fields, such as data range, field length, and default value, is extracted. Finally, the parsing results are stored as structured data in JSON format to build a protocol knowledge base for large models to call, ensuring that the subsequently generated test cases conform to the protocol rules.

[0097] S102. Construct a vulnerability description library, collecting vulnerability information related to industry protocols from vulnerability databases such as CVE (Common Vulnerabilities and Exposures), CNVD (China National Vulnerability Database), and CNNVD (China National Information Security Vulnerability Database), and extracting key information from the vulnerability descriptions, including:

[0098] Vulnerability name and number (e.g., CVE-2021-12345), affected protocol fields or function codes, and vulnerability type (e.g., buffer overflow, unauthorized access, command abuse);

[0099] Using field matching technology, fields in the vulnerability description are associated with field constraints in the protocol knowledge base. A "vulnerability context" is generated based on the content of the vulnerability description base for the test case generation module to refer to.

[0100] S103. Establish a real industrial flow database by collecting real industrial flow logs from industrial sites or simulation environments, including PCAP files, real-time flow records, or device response data.

[0101] Traffic analysis tools are used to parse traffic and extract protocol fields into structured data. The header and field content of the protocol application layer, such as function codes, starting addresses, and data lengths, are extracted. The traffic data is then converted into unlabeled text or hexadecimal format and stored in a knowledge base. This enriches the protocol instance data with a small number of labeled samples and a large number of unlabeled samples.

[0102] S104. Context fusion: Dynamically retrieve protocol-related field information based on the test task, combine the retrieved context information with the task instructions, generate dynamic prompts, and guide the model in subsequent multi-agent collaborative work.

[0103] S105.QLoRA fine-tuning quantizes the pre-trained LLM (Llama 3.1-8B) weights to 4-bit precision, significantly reducing memory requirements and computational overhead. Trainable low-rank matrices are inserted into the model's Transformer layers to adjust the model's adaptability to specific tasks.

[0104] W'=W+ΔW,ΔW=W A W B (1)

[0105] Where W is the original weight of the large model, W′ is the new weight obtained after fine-tuning with a low-rank matrix, and ΔW represents the weight adjustment term added during the fine-tuning process. A W B The decomposition of the low-rank matrix is ​​obtained through low-rank decomposition techniques, with the aim of optimizing the model with fewer parameter updates.

[0106]

[0107] in Θ is the loss function used to quantify the difference between the model's predictions and the actual labels. base P(y) represents the frozen parameters of the pre-trained model. i |x i ;Θ base ,ΔW) is the conditional probability, representing the probability given input x. i Under the given conditions, the model predicts the output y iThe probability of this is determined. With the basic weights of the model frozen, the low-rank matrix is ​​fine-tuned to enhance the model's adaptability to industrial protocol tasks (such as field parsing and test case generation). This fine-tuning is accomplished using a small amount of task data, ensuring resource efficiency.

[0108] S2. Capture real-time traffic and generate a test template, which is implemented by combining the communication interaction module and the seed generation agent. Specifically, in this embodiment, the process of capturing real-time traffic and generating a seed template is as follows:

[0109] S201. Use the traffic capture tool Scapy to capture real-time communication traffic between the system under test and external devices, parse application layer protocol data, extract key fields such as function code, address, data length, etc., and preliminarily identify the protocol type through protocol characteristics (such as port number or message format).

[0110] S202. Dynamic protocol knowledge retrieval: Extract field constraint information from protocol specification documents, including function code range, data length limits, etc.; retrieve protocol-related vulnerability descriptions from CVE and CNVD vulnerability databases; extract possible attack vectors; integrate protocol rules and vulnerability information; and generate context content for LLM to call.

[0111] S203. Combining chained reasoning technology, the protocol data parsing task is decomposed into multiple steps, and field extraction and verification are completed step by step through prompt word engineering. The rules retrieved by RAG are used to verify the extracted fields to ensure that the fields conform to the protocol specifications;

[0112] S204. Based on the retrieved protocol rules and field extraction results, construct a dynamic template for test case generation, ensure the protocol compliance and mutation adaptability of the seed, generate protocol-compliant test seed data, and output seed data in JSON format for use by the test case generation module.

[0113] S3. The test case generation proxy transforms high-quality seed data into diverse test cases that conform to the protocol rules. This is achieved through a combination of field mutation, structural mutation, and semantic mutation. Specifically, in this embodiment, the test case generation implementation steps are as follows:

[0114] S301. Obtain structured seed data containing protocol field information from the seed generation agent, preprocess the collected seed data and test strategy to ensure field integrity and strategy availability, verify whether the function code is within the scope specified by the protocol, and ensure that the mutation strategy is consistent with the protocol rules.

[0115] S302. To implement the seed mutation logic, the test case generation proxy employs three types of mutation methods: field mutation, structural mutation, and semantic mutation. Field mutation is achieved by randomly modifying field values ​​based on predefined rules, as shown below.

[0116] V′=V+ΔV,(3)

[0117] Where V′ is the output after field mutation, V is the original field value, and ΔV is the random or policy-driven increment;

[0118] Structural variations adjust the protocol structure by inserting invalid fields or deleting required fields, as shown below.

[0119] P′=P∪F insert \F delete (4)

[0120] Where P′ is the output after structural mutation, P is the original data input, and F insert F represents the set of invalid fields to be inserted. delete The set of required fields for deletion;

[0121] Semantic variation is modeled by introducing inconsistencies in protocol data, such as mismatched field relationships, and represented as...

[0122] S′=f(S,ΔS),(5)

[0123] Where S′ is the mutated semantic relation, S is the original semantic relation, and ΔS is the introduced anomaly;

[0124] S303. In the dynamic strategy adjustment phase, the mutation density is dynamically modified based on feedback information. The specific formula is as follows:

[0125] ρ=ρ0·(1+α·feedback_score), (6)

[0126] Where ρ is the adjusted mutation density, ρ0 is the initial mutation density, α is the feedback influence weight, and feedback_score is the severity of the detected anomaly. Dynamic prompts, combined with retrieval context and structured reasoning, guide the domain-enhanced large model to perform semantic mutation generation and mutation strategy optimization.

[0127] S304. Format the mutated test cases as JSON for injection into the system under test.

[0128] S4. The feedback analysis and strategy adjustment agent collects the response data of the system under test in real time, and dynamically adjusts the mutation strategy of the agent to generate test cases based on the real-time feedback, thereby realizing an automated and adaptive fuzzy testing closed loop. The specific steps are as follows:

[0129] S401. The data collected through the communication interface is analyzed by the feedback analysis and strategy adjustment agent to identify the trends and patterns of individual results, including normal responses, such as protocol-compliant confirmation messages (e.g., ACK), abnormal responses, i.e., unexpected error codes or delayed responses, and critical anomalies, such as serious problems like system crashes, timeouts, or resource consumption overload.

[0130] S402. Assign a weight to each feedback based on its anomaly type, response time, and resource usage, and calculate its severity score S. The specific formula is as follows:

[0131] S=w1·E+w2·T+w3·R,(7)

[0132] Where E is the anomaly type score (e.g., crash = 10, timeout = 8), T is the response time score (timeout = 8, latency = 3), R is the resource utilization score (CPU / memory peak = 6), and w1, w2, w3 are weights that can be adjusted according to the test objectives.

[0133] S403. Dynamically adjust the mutation density, priority field, and mutation direction based on the severity of anomalies identified in the feedback analysis to improve vulnerability discovery efficiency and test coverage. Specifically:

[0134]

[0135] Where ρ′ is the feedback-adjusted mutation density, ρ0 is the initial mutation density, and β is the adjustment coefficient. For standardized severity scoring, S max The total score for severity.

[0136] S404. Based on the anomaly type and feedback analysis results, adjust the priority of mutations, prioritize high-risk fields, and adjust the mutation direction to target potential vulnerability areas to ensure the efficiency and comprehensiveness of testing;

[0137] S405. When performing the next fuzzy cycle, refer to the feedback score and historical anomaly patterns to adjust the field priority when generating test cases, and ensure that high-risk areas are tested for mutations first.

[0138] The method described in this embodiment achieves intelligent and adaptive optimization of fuzz testing for industrial control protocols by constructing a multi-agent collaborative working mechanism. Specifically, this embodiment adopts a multi-layer architecture design, dynamically constructing a fuzz testing knowledge base by calling the RAG database through a seed generation agent, realizing feature fusion of semantic parsing and vulnerability intelligence for multiple protocols such as Modbus / TCP and S7Comm; generating diverse test cases based on the QLoRA-fine-tuned Llama3.1-8B model, supporting both deterministic and exploratory mutation modes; and establishing a ternary correlation analysis mechanism of "abnormal mode-device status-protocol features" through real-time assessment of vulnerability severity by sub-models. This embodiment, through the deep coupling of LLM multi-agent agents and industrial protocol fuzz testing, effectively solves the technical bottlenecks of insufficient protocol understanding and high mutation blindness in traditional fuzz testing in industrial control systems, thus building a solid security foundation for industrial networks throughout their entire lifecycle.

[0139] Example 2

[0140] This embodiment discloses an intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model, used to implement the intelligent fuzzy testing method for industrial control protocols based on a multi-agent large model described in Embodiment 1, such as... Figure 3 As shown, the intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model includes the following modules:

[0141] The domain-enhanced large model module performs 4-bit quantization fine-tuning on the Llama 3.1-8B model based on QLoRA technology, integrates the RAG engine, dynamically retrieves a 128GB industrial protocol knowledge base, and enables the LLM to have industrial protocol awareness capabilities. It is suitable for various industrial protocol environments, such as syntax tree parsing, test case generation optimization, and feedback adaptive adjustment of mainstream industrial control protocols such as Modbus / TCP, S7Comm, and Ethernet / IP.

[0142] The seed generation proxy module configures a dynamic constraint parser to extract field dependencies from the protocol specification in real time. Based on industry protocol specifications and a contextual knowledge base, it generates seed data that conforms to the protocol specification. It dynamically retrieves protocol constraints through RAG and combines this with LLM to generate protocol-aware input seeds, ensuring their legality and diversity.

[0143] The test case generation proxy module performs mutations on seed sequences based on a mutation policy library, including field mutations. It generates substitute values ​​and structural mutations based on the QLoRA fine-tuning model, inserts / deletes protocol fields and semantic mutations, and executes temporal logic perturbations. Through feedback-driven policy adjustments, it dynamically optimizes the mutation range and density to ensure high efficiency and vulnerability discovery capabilities.

[0144] The feedback analysis and strategy adjustment agent module analyzes the target system's response in real time and constructs a CVSS weighted score to quantify the severity of anomalies. For high-risk anomalies, the Proof-of-Concept (PoC) is immediately saved and the testing frequency is reduced; for medium-risk anomalies, the density of similar mutations is increased by 20%; for low-risk anomalies, the current strategy is maintained, and the mutation strategy is updated every 50 test cycles. Through clustering and weight analysis of runtime feedback, an anomaly pattern graph is established, associating protocol fields, device status, and vulnerability types.

[0145] The communication and interaction module enables real-time communication, task synchronization, and test case injection into the target system between various modules. ZeroMQ is used for high-speed test case injection; when the target system becomes unresponsive (timeout threshold 3 seconds), a backup link is automatically switched. The packet sending frequency is dynamically adjusted to avoid network congestion and ensure the stability of the testing process.

[0146] It should be noted that the workflow, operation content and technical achievements of the intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model disclosed in this embodiment 2 can be found in the specific steps shown in embodiment 1.

[0147] Example 3

[0148] This embodiment discloses a multi-dimensional evaluation method for fuzzy testing of industrial control protocols. By deploying it in a virtual-real combined attack and defense training range and integrating multi-dimensional performance evaluation indicators, a verification scheme is proposed for the intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model described in Embodiment 2. Figure 4 As shown, it includes the following technical features:

[0149] The model testing and inference platform includes a model training cluster and a deployment inference platform. The model training cluster consists of eight NVIDIA H100 GPUs (80GB VRAM), an Intel Xeon Gold 4310 CPU (12C24T@2.1GHz), and 128GB of DDR4 memory. Software-wise, it uses Ubuntu 20.05, PyTorch 2.0 + Hugging Face Transformers 4.28, and model fine-tuning employs QLoRA technology with 4-bit NF4 quantization. The deployment inference platform consists of two NVIDIA RTX 4090 GPUs (24GB VRAM), an Intel Core i9-14900K CPU (24C32T@2.4GHz), and 64GB of DDR5 memory. It uses Ubuntu 20.05, with Scapy 2.5 as the network engine and a Redis 7.0 cluster for state management to improve protocol reconstruction accuracy and reduce query latency.

[0150] The virtual-real integrated testing platform comprises a physical layer and a virtual layer. The physical layer utilizes a typical energy enterprise attack and defense exercise range from the Liaoning Provincial Key Laboratory of Information Security in the Petrochemical Industry. This range integrates mainstream commercial PLCs, including Siemens S7-300, S7-1200 series, Rockwell RSLogix5000 series, Mitsubishi FX series, and Emerson VersaMax series, providing a comprehensive real-world industrial control system testing platform. The virtual layer employs PLCSIM Advanced's full-state machine simulation, building an industrial ring network based on Mininet to achieve a high-coverage, low-latency industrial control protocol and network simulation testing environment.

[0151] The baseline testing method was compared with three fuzzing tools, representing different levels of protocol understanding and automation capabilities, under the same testing platform and environment:

[0152] Peach, a widely used fuzzing framework, focuses on the variation of input data. Based on Peach's guiding principles, models were created for each target service, defining data formats and state transitions.

[0153] NCMFuzzer is a tool specifically designed for the ICS protocol. It can handle packet structures such as Modbus / TCP and S7, integrates domain-specific knowledge to discover protocol vulnerabilities, and provides configuration of captured network packets and related field constraints.

[0154] ChatAFL, an LLM-based fuzzing tool, employs a single-agent architecture and improves upon it by leveraging its built-in hint mechanism and using seed data from industrial traffic logs; all the baseline fuzzing tools mentioned above are configured according to recommended practices to ensure the fairness of the comprehensive evaluation.

[0155] Multidimensional performance evaluation metrics are used to assess the fuzzing capabilities and test case quality of various tools in industrial network environments based on two key technical indicators: fuzzing effectiveness and test case richness.

[0156] The fuzzing effectiveness is comprised of the Test Case Pass Rate (TCPR) and the Number of Exception Triggers (ETN). TCPR measures how many of the generated test cases are successfully processed by the system under test without being rejected or causing parsing errors. Its definition is as follows:

[0157]

[0158] The number of passed test cases refers to the number of test cases successfully accepted by the SUT, and the total number of test cases is the total number of all generated test cases. A higher TCPR usually indicates that the generated test cases follow the basic syntax rules of the protocol, thus providing a meaningful opportunity to trigger a system response.

[0159] Exception trigger count (ETN) i This reflects the framework's ability to detect potential vulnerabilities (such as crashes or abnormal system states). In the standard experimental design, three rounds of fuzz testing are conducted, each lasting 24 hours, with sampling every 3 hours. ETN is defined as the total number of crashes triggered in each round of testing:

[0160] ETN i = (Number of crashes triggered in the i-th test round), i = 1, 2, 3 (10)

[0161] The richness of test cases is composed of coverage and Shannon entropy. Coverage represents the inclusion of valid field-value combinations in the seed database according to the protocol specification. Its definition is as follows:

[0162]

[0163] The number of covered combinations is the number of different field-value pairs existing in the seed library, and the total number of combinations is the number of all possible field-value pair combinations defined by the protocol specification.

[0164] Shannon entropy H(X) quantifies the variability of field values ​​in test cases, and the formula is as follows:

[0165]

[0166] Where X is a random variable, representing the field value encountered in the mutation test set, p(x) i ) is the observation of a specific value x i The probability of H(X). A larger H(X) value indicates a wider distribution of field values, suggesting that the mutation explored a more diverse range of possibilities.

[0167] This embodiment demonstrates the advantages of the multi-agent large model fuzzing system in surpassing the limits of traditional fuzzing, domain characteristics, and compared to benchmark methods through quantitative analysis and interpretive insights of key evaluation indicators.

[0168] The multi-agent large-model fuzzing system described above achieved a TCP pass rate (TCPR) of 88%-92% on three mainstream industrial control protocols: Modbus / TCP, S7, and Ethernet / IP, significantly outperforming Peach (74%), NCM Fuzzer (83%), and ChatAFL (81%). In three rounds of 24-hour fuzzing, the system induced approximately 22 crashes on a commercial PLC running Modbus / TCP, exceeding NCM Fuzzer (17 crashes), ChatAFL (15 crashes), and Peach (12 crashes), demonstrating the system's ability to discover critical system vulnerabilities.

[0169] Regarding the richness of test cases, the multi-agent large-model fuzzing system achieves over 90% coverage, higher than Peach, ChatAFL (40%-60%), and NCM Fuzzer (83%). The Shannon entropy value is 4.2-4.6, better than Peach, NCM Fuzzer, and ChatAFL (3.1-4.1). Experimental results show that the system achieves comprehensive coverage while ensuring high variability, thus guaranteeing in-depth exploration of industrial control protocols.

[0170] In subsequent fuzzing applications, the multi-agent large-model fuzzing system independently discovered several industrial control protocol vulnerabilities, such as... Figure 5 As shown, this includes known CVEs and three zero-day vulnerabilities, none of which were detected by benchmark tools. One of the zero-day vulnerabilities, CNVD-2024-16009, has been identified and reported to the China National Vulnerability Database.

[0171] It should be noted that the above-described embodiments are merely specific examples of the present invention, used to help understand the technical solution and core ideas of the present invention, and do not constitute any limitation on the scope of protection of the present invention. Any equivalent substitutions, combination optimizations, or adaptive modifications made by those skilled in the art to the system architecture, parameter configurations, or implementation steps described in the embodiments based on the technical concept of the present invention after reading the present invention specification should be included within the scope of protection defined by the claims of the present invention.

[0172] The embodiments of this invention are not limited to those described in the specification, and their technical features can be combined or separated to form new technical solutions. Any reasonable modifications made in accordance with the core design concept of this invention, such as the expansion of industrial control protocol types, adaptation of hardware platforms, or replacement of algorithm components, are within the scope of protection of this invention. The actual scope of protection of this invention should be determined by the claims.

Claims

1. A method for intelligent fuzzy testing of industrial control protocols based on a multi-agent large model, characterized in that, The method includes the following steps: Building a Domain-Enhanced Large Model: By integrating Retrieval Enhanced Generation (RAG) and Low-Rank Quantization Adaptation (QLoRA) techniques, a large language model is built that can parse industry protocol constraints, optimize test case generation, and perform adaptive feedback adjustments. Seed generation agent: Based on industry protocol specifications, it uses RAG technology to dynamically retrieve protocol constraints and generates seed data that conforms to the protocol specifications through multi-agent collaboration; Test case generation agent: Utilizing seed data and combined with a large language model fine-tuned by QLoRA, it performs field mutation, structural mutation, and semantic mutation to generate diverse protocol test cases; Feedback Analysis and Strategy Adjustment Agent: Analyzes the response data of the target system in real time, and adjusts the testing strategy through cluster analysis and anomaly scoring to optimize the coverage of test cases and vulnerability discovery capabilities; Communication and interaction module: intercepts real-time data, metadata and payload of industrial control protocols, and ensures the efficiency, stability and scalability of fuzz testing through real-time communication and task synchronization between multiple agents.

2. The intelligent fuzzy testing method for industrial control protocols based on a multi-agent large model according to claim 1, characterized in that, The domain-enhanced large model includes: The RAG module is used to dynamically extract protocol constraint information from knowledge bases such as industry protocol specifications and vulnerability databases (CVEs), and to provide contextual guidance during seed generation and test case generation. The QLoRA fine-tuning module is used to optimize large language models in resource-constrained environments. It improves the adaptability of protocol tasks by fine-tuning the model weights through a low-rank matrix.

3. The intelligent fuzzy testing method for industrial control protocols based on a multi-agent large model according to claim 1, characterized in that, The seed generation agent includes the following functional sub-modules: The protocol knowledge dynamic retrieval submodule dynamically retrieves relevant rules and constraints from the industry protocol knowledge base via RAG; The field extraction and verification submodule uses chained reasoning (CoT) to parse protocol data packet fields step by step, ensuring that the parsing logic is clear and the results are reliable. The dynamic template generation submodule automatically generates test seed templates based on the field extraction results and the protocol rules retrieved by RAG.

4. The intelligent fuzzy testing method for industrial control protocols based on a multi-agent large model according to claim 1, characterized in that, The test case generation agent includes the following functional sub-modules: The input acquisition submodule obtains structured seed data from the seed generation agent and obtains test strategy priority and mutation intensity adjustment parameters from the feedback analysis agent. The mutation logic submodule performs field mutations, structural mutations, and semantic mutations based on the input data to generate diverse test cases; The output generation submodule formats the mutated test cases into JSON format for subsequent fuzz testing.

5. The intelligent fuzzy testing method and system for industrial control protocols based on a multi-agent large model according to claim 1, characterized in that, The feedback analysis and strategy adjustment agent includes the following functional sub-modules: The feedback data acquisition and classification submodule collects response data from the system under test and classifies the feedback into normal response, abnormal response and critical anomaly based on predefined criteria. The abnormal behavior analysis submodule uses a weighted scoring system to quantify anomalies and extracts potential vulnerability patterns from historical test data. The dynamic strategy adjustment submodule dynamically adjusts mutation density and direction based on feedback analysis results to improve test coverage and attack power.

6. The intelligent fuzzy testing method for industrial control protocols based on a multi-agent large model according to claim 1, characterized in that, The communication interaction module includes the following functional sub-modules: The real-time traffic capture submodule intercepts real-time data packets of the target protocol and extracts protocol metadata and payload. The test case injection submodule enables real-time injection and response monitoring of test cases, ensuring that the injected test cases can be seamlessly integrated into the protocol communication link. The multi-agent coordination submodule, based on task queues and asynchronous message passing mechanisms, enables efficient data exchange and task collaboration among seed generation agents, test case generation agents, and feedback analysis agents.

7. An intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model, characterized in that, The system includes: The domain-enhanced large model module is used to build large language models through RAG and QLoRA technologies. It has industrial protocol awareness capabilities and can realize the parsing and test case generation of various industrial protocols. The seed generation agent module generates seed data that conforms to the protocol specifications by using the protocol knowledge base and context information; The test case generation proxy module generates efficient test cases based on seed data and feedback analysis results through field, structure, and semantic variations. The feedback analysis and strategy adjustment agent module analyzes the target system's response in real time and adjusts the testing strategy based on the feedback to improve vulnerability discovery capabilities. The communication and interaction module supports task synchronization and test case injection between various agent modules, ensuring efficient and stable testing processes.

8. The intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model according to claim 7, characterized in that, The aforementioned domain-enhanced large model includes a RAG module and a QLoRA fine-tuning module. The RAG module is used to dynamically extract protocol constraints and industrial control vulnerability database information. The QLoRA module reduces computational resource requirements by freezing the basic parameters of the LLM and training the inserted low-rank matrix, thus enabling the large model to adapt to low-resource environments.

9. The intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model according to claim 7, characterized in that, The seed generation agent includes a protocol knowledge dynamic retrieval submodule, a field extraction and verification submodule, and a dynamic template generation submodule; By combining RAG technology, relevant rules and constraints are dynamically retrieved from protocol specification documents, CVE vulnerability databases, and real traffic logs to provide contextual information for generating seed data that conforms to the protocol specification. The test case generation agent includes an input collection submodule, a mutation logic submodule, and an output generation submodule. Combined with CoT technology, it generates diverse and protocol-compliant test cases to ensure comprehensive discovery of protocol vulnerabilities.

10. The intelligent fuzzy testing system for industrial control protocols based on a multi-agent large model according to claim 7, characterized in that, The feedback analysis and strategy adjustment agent includes a feedback data collection and classification submodule, an abnormal behavior analysis submodule, and a dynamic strategy adjustment submodule. It quantifies the severity of abnormal responses through a weighted scoring mechanism and adjusts testing strategies based on the scoring results to ensure maximum test coverage and vulnerability discovery capabilities. The communication interaction module includes a real-time traffic capture submodule, a test case injection submodule, and a multi-agent coordination submodule. It intercepts and processes real-time data of the target protocol using the traffic capture tool Scapy, and further employs the ZeroMQ message passing mechanism. Through task queues and asynchronous message passing, it achieves real-time communication, task synchronization, and test case injection between multiple agents, ensuring an efficient testing process and system stability.

Images