A multi-model collaboration system and method for industrial software protocol fuzz testing

Abstract

The present invention provides a multi-model collaboration system and method for fuzz testing of industrial software protocols, including: a large model for analyzing industrial software network protocols, which is used to obtain the semantic information of industrial software protocols and generate a fuzz test seed queue in combination with prompt words; a large model for generating test samples, which is used to establish a mutation strategy according to the constraints in the fuzz test of the target software, and mutate the fuzz test seeds through knowledge reasoning to generate test samples; a large model for target monitoring and feedback, which is used to perform fuzz testing on the target software service according to the test samples and perform abnormal monitoring at the same time; a multi-model collaboration Agent, which is used to complete model scheduling, result integration and process control and test strategy optimization. The present invention not only improves the test effect, but also greatly reduces the work intensity of testers and reduces labor costs; it can give full play to the testing capabilities of multiple models, effectively improve the coverage of fuzz testing, and improve the efficiency of fuzz testing.

Description

Technical Field

[0001] The present invention relates to the field of industrial software, and in particular to a multi-model collaboration system and method for fuzzy testing of industrial software protocols. Background Art

[0002] The present invention relates to the fact that in recent years, the communication protocols between various industrial devices and systems have become increasingly complex. These protocols not only contain rich functional characteristics, but also often have complex state transition logic and boundary conditions. This has brought great challenges to protocol security assessment, and traditional fuzz testing methods have been unable to meet the increasingly complex and diverse security requirements of industrial protocols.

[0003] The current industrial protocol fuzz testing method, which is mainly oriented towards improving coverage, has a certain degree of blindness and randomness. Although it can increase the detection range of the test and touch more code paths, it is difficult to explore potential vulnerability areas in a targeted manner. Even if a large number of anomalies can be found, the proportion of truly valuable security vulnerabilities is low. Industrial protocols are usually more complex and have a huge state space, which is also a bottleneck that hinders the improvement of the execution efficiency of fuzz testing. The discovery of a large number of low-risk, non-critical vulnerabilities cannot effectively improve the security of the overall system.

[0004] The large language model is an artificial intelligence technology that has emerged in recent years. It can generate human-readable natural language text by learning massive text data. The powerful language generation ability of the large language model brings new possibilities to fuzz testing. Systems represented by Fuzz4All and ChatAFL will use the powerful semantic understanding and generation capabilities of the large language model to generate effective input data, and continuously iterate and improve the performance of the language model through feedback information, effectively improving the efficiency of protocol fuzz testing. Fuzz4All uses a pre-trained general language model and continuously optimizes the generation of test cases through interactive learning with human testers, thereby covering a wider input space. However, there is no modeling and processing for protocol states, and it is impossible to generate test cases covering various state transitions of the state machine in a targeted manner. ChatAFL combines the AFL fuzz testing engine and uses the GPT model to generate targeted test cases for in-depth testing of specific protocols or programs. ChatAFL focuses more on specific goals, but its adaptability is relatively narrow, and its generalization ability is insufficient to adapt to complex scenarios of industrial software.

[0005] In general, network protocol fuzz testing technology based on large language models is constantly developing and maturing, and may become an important tool in the field of software testing in the future. However, to truly apply it to industrial-grade software, it is still necessary to further improve the ability to understand specific protocols and enhance the pertinence and interpretability of test cases. Summary of the invention

[0006] In view of the problems existing in the prior art, a multi-model collaborative system and method for industrial software protocol fuzz testing are provided, which improves the efficiency of industrial software protocol fuzz testing by performing fuzz testing on industrial software protocols through multi-model collaboration.

[0007] A first aspect of the present invention provides an industrial software protocol fuzz testing multi-model collaboration system, comprising:

[0008] Industrial software network protocol analysis model, used to obtain the semantic information of industrial software protocols and generate fuzz test seed queues in combination with prompt words;

[0009] The test sample generation model is used to establish mutation strategies according to the constraints in the target software fuzz test, and mutate the fuzz test seeds through knowledge reasoning to generate test samples;

[0010] The target monitoring and feedback model is used to perform fuzz testing on the target software service according to the test samples, monitor the abnormalities of the tested target software service, and feed back the monitoring results to the multi-model collaboration agent;

[0011] The multi-model collaborative agent is used to determine the execution order and parameter configuration of each model according to the overall test goal, dynamically adjust the fuzzy test strategy based on the output of the target monitoring and feedback large model, monitor the execution status of the entire test process and intervene and adjust when encountering abnormal situations, and summarize the output results of each model and perform data fusion and analysis to form the final test report.

[0012] As a preferred solution, the working process of the industrial software network protocol analysis large model includes:

[0013] Obtaining a protocol source code dataset, a protocol standard document dataset, and a protocol traffic dataset;

[0014] Filter the protocol source code data to obtain relevant codes, split the codes to obtain code blocks, and finally extract key information from the code blocks as feature vectors;

[0015] Split and chunk the protocol standard document dataset, obtain standard text chunks and protocol paradigms, and input them into the big model;

[0016] For the protocol traffic data set, the packets are segmented based on the session granularity to remove the interference information in the session. The protocol session packet sequence is then provided as input to the large model, which extracts the protocol information from the session sequence.

[0017] The large model learns from formalized protocol elements including the protocol's feature vectors, protocol paradigms, and session sequences, infers the semantic constraints between protocol elements, and generates diverse input data based on prompt words to cover different paths and boundary cases of the protocol, generating the optimal fuzz testing seed sequence.

[0018] As a preferred solution, in the test sample generation large model, a single strategy or a combination of multiple strategies is selected to mutate the seed, wherein the mutation strategies include bit / byte operations, digital mutations, string mutations, checkpoint bypasses, and injection attacks.

[0019] As a preferred solution, the target monitoring and feedback model inputs the generated test cases into the target system, performs status tracking, coverage statistics and anomaly monitoring on the test process, captures various system events during the execution process in real time, and outputs the test results.

[0020] As a preferred solution, the working process of the multi-model collaborative agent includes: task decomposition: splitting the fuzzy test target into subtasks that can be executed by each model; task configuration: configuring parameters for each subtask; task distribution: distributing each subtask together with the configuration parameters to the corresponding model for execution.

[0021] As a preferred solution, the working process of the multi-model collaboration Agent also includes: summarizing the output results of each model and feeding back test information; and performing correlation analysis and failure mode analysis on the execution results of each model to determine whether potential vulnerabilities are found.

[0022] As a preferred solution, the working process of the multi-model collaborative agent also includes: timely adjusting the seed scheduling strategy according to the feedback information in the test, screening the seeds from the seed queue, and selecting seed samples that better meet the requirements; adjusting the probability of the mutation strategy according to the coverage information to cover more boundary conditions and improve the coverage of the fuzzy test; filtering the seeds according to the results of the test feedback, and selecting seeds that trigger more exceptions to add to the seed queue.

[0023] The second aspect of the present invention provides an industrial software protocol fuzzy testing multi-model collaboration method, which is implemented based on the industrial software protocol fuzzy testing multi-model collaboration system described in the first aspect, and includes:

[0024] Industrial software network analysis steps: Analyze the data related to the industrial software protocol by building an industrial software network analysis model to generate a seed queue for fuzz testing;

[0025] Test case generation step: Generate a large model by building test cases, and mutate the input seed samples to generate new seeds based on the semantic understanding ability of the large model;

[0026] Abnormal monitoring step: by building a large abnormal monitoring model, the target under test is monitored for abnormalities and the monitoring results are obtained;

[0027] Multi-model collaboration steps: Determine the execution order and parameter configuration of each model according to the overall test objectives, and issue task instructions to perform fuzz testing on the target software service; dynamically adjust the fuzz testing strategy based on the output of the target monitoring and feedback large model; summarize the output results of each model, perform data fusion and analysis to form the final test report.

[0028] As a preferred solution, the industrial software network analysis step specifically includes:

[0029] Through the industrial software network analysis large model, formalized protocol elements are obtained from the protocol source code dataset, the protocol standard document dataset, and the protocol traffic dataset for learning, and the semantic constraints between the protocol elements are inferred. Based on the prompt words, diversified input data is generated to cover different paths and boundary conditions of the protocol, and the optimal fuzz test seed samples are generated.

[0030] As a preferred solution, the multi-model collaboration step specifically includes:

[0031] Task scheduling: split the fuzz testing target into subtasks that can be executed by each model; configure parameters for each subtask; send each subtask together with the configuration parameters to the corresponding model for execution;

[0032] Result integration and process control: Summarize the output results of each model and provide feedback on the test information; perform correlation analysis and failure mode analysis on the execution results of each model to determine whether potential vulnerabilities are found; and monitor the execution status of the entire test process and intervene and adjust when abnormal situations are encountered;

[0033] Test strategy optimization: timely adjust the seed scheduling strategy according to the feedback information in the test, screen the seeds from the seed queue, and select seed samples that better meet the requirements; adjust the probability of the mutation strategy according to the coverage information to cover more boundary conditions and improve the coverage of fuzz testing; filter the seeds according to the results of the test feedback, and select seeds that trigger more anomalies to add to the seed queue.

[0034] Compared with the prior art, the beneficial effects of adopting the above technical solution are:

[0035] 1. The multi-model collaboration solution based on the Agent framework of the present invention makes full use of the excellent reasoning and logic processing capabilities of the big model. The big model has excellent performance in natural language understanding, knowledge reasoning, etc., and can deeply understand the complex semantics and logical structure of industrial software protocols. By integrating the big model into the fuzzy testing framework, the detection capabilities of protocol boundary conditions and abnormal situations can be greatly improved, and potential security vulnerabilities and design defects can be discovered more accurately. Compared with traditional rule-based or pattern matching testing methods, big model-driven fuzzy testing is more flexible and powerful, and can quickly adapt to changes and updates in the protocol, which not only improves the testing effect, but also greatly reduces the workload of testers and reduces labor costs.

[0036] 2. The Agent framework of the present invention adopts a multi-model collaborative design. By organically combining these models through the Agent framework, the testing capabilities of multiple models can be fully utilized, the coverage of fuzz testing can be effectively improved, and the efficiency of fuzz testing can be improved. The Agent framework can monitor the operating status of each model in real time, dynamically schedule resources, and ensure the efficient and orderly progress of the testing process. The framework has strong adaptability and can dynamically adjust the testing strategy and parameters according to the feedback information obtained during the testing process. Through this adaptive mechanism, the testing process is continuously optimized and improved, and the testing efficiency and accuracy are continuously improved. Compared with traditional fuzz testing solutions, this adaptive framework can better cope with the ever-changing characteristics of complex protocols and improve the overall test adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the multi-model collaborative system for industrial software protocol fuzz testing proposed in the present invention.

[0038] Figure 2 This is a schematic diagram of the working of the industrial software network protocol analysis large model in the industrial software protocol fuzzy testing multi-model collaborative system proposed by the present invention.

[0039] Figure 3 A schematic diagram of a large model for generating test samples in the multi-model collaborative system for fuzz testing of industrial software protocols proposed in the present invention.

[0040] Figure 4 This is a schematic diagram of the target monitoring and feedback large model in the industrial software protocol fuzzy testing multi-model collaborative system proposed by the present invention.

[0041] Figure 5 This is a schematic diagram of the multi-model collaboration agent in the multi-model collaboration system for industrial software protocol fuzz testing proposed by the present invention. DETAILED DESCRIPTION

[0042] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar modules or modules with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as limitations on the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents that fall within the spirit and connotation of the appended claims.

[0043] The present invention mainly solves the following problems:

[0044] 1. Big model’s knowledge understanding and inference of industrial software network protocols

[0045] Although large language models perform well in a wide range of fields such as natural language processing, they often have some limitations when dealing with network protocols in the field of industrial software. The main limitation is the lack of in-depth understanding of specific protocol details and boundary conditions, and difficulty in adapting to complex protocol state structures and state transition logic. This makes it difficult for large models to be directly applied to scenarios such as fuzz testing in the field of industrial software. Therefore, it is necessary to enhance the capabilities in industrial software protocols to help the model better understand and process related knowledge restoration and infer protocol information to generate more effective fuzz testing seed samples.

[0046] 2. Test case generation and anomaly monitoring for industrial software protocol fuzz testing using large models

[0047] Traditional protocol fuzz testing methods, such as AFLNET, use a coarse-grained evolutionary algorithm to select the state of the target to be tested based on the state information saved in the fuzz test. This method relies too much on code coverage and ignores the potential relationship between states. In terms of the feedback mechanism of coverage, coverage evaluation is limited to a single dimension, and targeted programming is required for different protocol coverage statistics, which is difficult to reuse. In traditional fuzz testing, fast and small test cases are often pursued to cover most parsing codes as quickly as possible. However, in protocol fuzz testing, new data packets can only be covered after a certain number of data packets are sent, which requires longer and more appropriate test cases. Traditional test case generation methods will waste more testing time on some easily triggered cases.

[0048] How to use the powerful semantic understanding, dynamic reasoning and knowledge modeling capabilities of large models to solve the shortcomings of traditional methods in protocol status tracking, coverage guidance feedback and test case generation, effectively guide test case generation and optimization, and improve the fuzz testing capabilities of industrial software protocols.

[0049] 3. Build an Agent Framework for Multi-model Collaboration in Industrial Software Protocol Fuzz Testing

[0050] How to build a locally fine-tunable multi-model collaboration tool learning agent framework, coordinate the interaction and collaboration between various models, ensure the orderly progress of the testing process, and dynamically schedule the work of each model according to the test feedback information to promote the improvement of fuzz testing efficiency.

[0051] Based on this, please refer to Figure 1 The embodiment of the present invention proposes an industrial software protocol fuzzy testing multi-model collaboration system, which mainly includes an industrial software network protocol analysis model, a test sample generation model, a target monitoring and feedback model, and a multi-model collaboration Agent. Among them, the industrial software network protocol analysis model uses the standard documents (RFC) of the industrial software protocol, the source code of the protocol implementation, and the data packets of the protocol communication as data samples to obtain the semantic information of the protocol (including key information such as message type, field definition, state machine, etc.) to generate a fuzz test seed queue; the test sample generation model is used to establish a mutation strategy according to the constraints in the target software fuzz test, and mutate the fuzz test seeds through knowledge reasoning to generate test samples; the target monitoring and feedback model is used to perform fuzz testing on the target software service according to the test samples, and at the same time perform abnormal monitoring on the tested target software service. The prompt words of the model and the strategy for test case generation can be adjusted based on the feedback of the monitoring result information; the multi-model collaboration agent, as the "brain" of the entire system, is used to determine the execution order and parameter configuration of each model according to the overall test goal, dynamically adjust the fuzz test strategy based on the output of the target monitoring and feedback model, monitor the execution status of the entire test process, intervene and adjust when encountering abnormal situations, and summarize the output results of each model and perform data fusion and analysis to form the final test report.

[0052] The working of each model is described in detail below.

[0053] Please refer to Figure 2The industrial software network protocol analysis model mainly automatically extracts various key protocol knowledge points from the protocol data, extracts the overall architecture of the protocol, message format, state machine and other structural information, analyzes the protocol data packet, extracts the fields of the actual message, value range and other dynamic information, extracts the data processing logic, error handling process and other details, and automatically generates the seed samples for fuzz testing. After obtaining the protocol source code data set, protocol standard document data set, and protocol traffic data set, the data must first be preprocessed to remove irrelevant noise, format errors and redundant information to ensure data quality and consistency. Secondly, the data set is divided into blocks, and the cleaned data set is divided into multiple small blocks. Each block focuses on a specific information topic, such as message type, state transition, etc., to facilitate more accurate identification of key structures and elements in the protocol. Finally, the industrial software protocol information inferred by the large model is guided by the input prompt word to generate the original seed for fuzz testing.

[0054] In this embodiment, the data of the industrial software protocol mainly includes three types: industrial software protocol standard documents (RFC), open source implementation source code of industrial software protocols, and data packets of industrial software protocols. Industrial software protocol standard documents (RFC) include general network protocols (such as TCP / IP, HTTP, MQTT, etc.) and various industrial field bus protocols (such as PROFIBUS, MODBUS, CAN bus, etc.). The open source implementation source code of the protocol, such as the open source substation automation system based on IEC 61850, the open source PLC controller based on MODBUS, and other industrial protocols have corresponding open source implementation codes. The data packets in the protocol communication process record the actual encoding format of the protocol message, the interaction process and other details, which helps the large model to better understand the working principle of the protocol.

[0055] Furthermore, in this embodiment, the acquired protocol source code data set, protocol standard document data set, and protocol flow data set are processed as follows:

[0056] For the protocol source code dataset, the main work is to filter the protocol source code data to obtain relevant codes, then split the codes to obtain code blocks, and finally extract key information from the code blocks as feature vectors.

[0057] For the protocol standard document dataset, the protocol standard document dataset is split and divided into blocks, and the standard text blocks and protocol paradigms are obtained and input into the large model. Among them, the blocks include request messages, response messages, state transition messages and related context information during protocol communication, and the protocol paradigm includes the message type, format field description, state sequence, etc. of the protocol.

[0058] For the protocol traffic data set, the messages are segmented at the session granularity to remove interference information such as retransmission, disorder, and fragmentation in the session. The session message sequence of a certain protocol is provided as input to the large model, and the large model extracts the protocol information from the session sequence.

[0059] Finally, the large model learns from formalized protocol elements such as the protocol's feature vectors, protocol paradigms, and session sequences, and infers the semantic constraints between protocol elements. The model generates diverse input data based on prompts to cover different paths and boundary conditions of the protocol and generate optimal fuzz testing seed samples.

[0060] Please refer to Figure 3 The test sample generation large model is mainly used for test cases. In this embodiment, the input seed sample is mutated to generate new seeds based on the semantic understanding ability of the large model. Generate semantically similar seeds while maintaining executable conditions, providing richer materials for subsequent fuzz testing.

[0061] Specifically, in this embodiment, by analyzing the target program service, referring to the program code, protocol RFC documents or data packets, etc., the input data type, format and constraints of the target system are obtained, and according to the selected seed data characteristics, the corresponding mutation strategy is selected, and the program is written to automatically generate a large number of mutated test cases.

[0062] In this embodiment, the mutation strategies used include bit / byte operations (such as randomly flipping bits, replacing / deleting / adding bytes, etc.), numerical mutation (such as increasing, decreasing or multiplying integers / floating-point numbers by random numbers), string mutation (such as truncation, splicing, replacing characters, etc.), injection attacks (such as SQL injection, cross-site scripting (XSS), path traversal, etc.), checkpoint bypass (bypassing input verification by using Unicode encoding, etc.), etc. The above compilation strategies are commonly used in the embodiments of the present invention, and other mutation strategies can also be used. In actual applications, a combination of one or more mutation strategies can be selected according to the target program service to mutate the seed.

[0063] Please refer to Figure 4 The target monitoring and feedback model is mainly used to input the generated test cases into the target system, track the state of the test process, perform coverage statistics, and monitor anomalies. It captures various system events in the execution process in real time, such as memory access, function calls, etc., and monitors whether the target program crashes, hangs, and other abnormal behaviors. If the system crashes, behaves abnormally, or has incorrect output, security vulnerabilities may be discovered. Finally, the test results are output. The strategy of the large model's prompt word enhancement and test case generation can be adjusted according to the monitoring result information feedback to improve the efficiency of fuzz testing.

[0064] Please refer to Figure 5 The multi-model collaboration agent is the core of the whole system. It is mainly responsible for coordinating the large model of industrial software network protocol analysis, the large model of test sample generation, and the large model of target monitoring and feedback, so that the whole fuzz testing process can run efficiently. It mainly performs the following tasks:

[0065] (I) Model scheduling: Determine the execution order and parameter configuration of each model according to the overall test goal, and issue task instructions. Model scheduling is the basic function of the Agent. It needs to determine the execution order and parameter configuration of each model according to the overall test goal, and issue the corresponding task instructions. It mainly includes key steps such as task decomposition, task configuration, and task issuance: (1) Task decomposition: The Agent needs to decompose the overall fuzz test goal into a series of executable subtasks, including generating fuzz test protocol semantics for a certain interface, generating corresponding test cases based on specific semantics, and performing target system monitoring for a batch of test cases. For each subtask, the Agent selects a suitable model to execute. (2) Task configuration: The Agent framework also needs to configure appropriate parameters for each subtask to maximize the overall test effect. (3) Task issuance: The Agent framework sends each subtask together with the corresponding configuration parameters to the corresponding model for execution. In order to make full use of computing resources, these subtasks can be executed in parallel. Through the above steps, the Agent framework can arrange the work of each model in an orderly manner to ensure the efficient operation of the entire test process.

[0066] (II) Result integration and process control: Agen is also responsible for summarizing the output results of each model and timely feedback of test information, including abnormal information during the test, changes in coverage, etc. It is also necessary to perform data fusion and analysis to form test results. Agent conducts in-depth analysis of the execution results to determine whether potential vulnerabilities have been found.

[0067] The analysis mainly includes correlation analysis and failure mode analysis. Correlation analysis is to conduct correlation analysis on various system events captured during the execution process to find out the potential connections that lead to abnormal behaviors, so as to better understand the root causes of the vulnerabilities. Fault mode identification mainly attempts to identify some typical failure modes in the execution results, such as memory out-of-bounds errors and division by zero errors. Learn the knowledge of failure modes and automatically detect whether these known failure features exist in the execution results to accurately determine whether a vulnerability has been found. Through the combination of the above two analysis methods, various abnormal situations in the execution process of fuzzy input are comprehensively explored to provide a basis for the next step of feedback update.

[0068] (III) Test strategy optimization: Based on the output of the target monitoring and feedback model, dynamically adjust the fuzz test strategy, such as modifying the protocol semantic generation rules and test case generation templates, to improve the efficiency of vulnerability discovery. Test strategy optimization is to timely adjust the seed scheduling strategy, seed screening strategy and seed mutation strategy execution probability according to feedback information during the test process. According to the feedback information in the test, timely adjust the seed scheduling strategy, screen the seeds from the seed queue, and select more suitable seed samples. According to the coverage information, adjust the probability of the mutation strategy to cover more boundary conditions and improve the coverage of the fuzz test. According to the test feedback results, filter the seeds and select the seeds that trigger more anomalies to add to the seed queue. The fuzz test process is a dynamic cycle. The agent continuously analyzes the feedback data, makes strategy optimization decisions, and updates the model parameters to ensure that the overall test effect is continuously improved. Through this feedback-based test strategy optimization mechanism, the agent can help the entire multi-model collaboration framework to achieve an adaptive fuzz test process and greatly improve the ability to discover security vulnerabilities.

[0069] The present invention makes full use of the excellent reasoning and logic processing capabilities of the big model. The big model has excellent performance in natural language understanding, knowledge reasoning, etc., and can deeply understand the complex semantics and logical structure of industrial software protocols. By integrating the big model into the fuzzy testing framework, the detection capability of the protocol boundary conditions and abnormal situations can be greatly improved, and potential security vulnerabilities and design defects can be discovered more accurately. Compared with the traditional test method based on rules or pattern matching, the fuzzy test driven by the big model is more flexible and powerful, and can quickly adapt to the changes and updates of the protocol, which not only improves the test effect, but also greatly reduces the work intensity of the testers and reduces the labor cost. At the same time, the present invention adopts a multi-model collaborative design, and organically combines these models through the Agent framework, which can give full play to the testing capabilities of multiple models, effectively improve the coverage of fuzzy testing, and improve the efficiency of fuzzy testing. The Agent framework can monitor the running status of each model in real time, dynamically schedule resources, and ensure the efficient and orderly progress of the test process. The framework has strong adaptability and can dynamically adjust the test strategy and parameters according to the feedback information obtained during the test process. Through this adaptive mechanism, the test process is continuously optimized and improved, and the test efficiency and accuracy are continuously improved. Compared with traditional fuzz testing solutions, this adaptive framework can better cope with the ever-changing characteristics of complex protocols and improve the overall test adaptability.

[0070] Furthermore, an embodiment of the present invention also provides an industrial software protocol fuzz testing multi-model collaboration method, including:

[0071] Step 100, industrial software network analysis step: construct an industrial software network analysis model to analyze data related to the industrial software protocol and generate a seed queue for fuzz testing;

[0072] Step 200, test case generation step: construct a test case generation large model, and mutate the input seed sample to generate a new seed based on the semantic understanding ability of the large model;

[0073] Step 300, abnormality monitoring step: construct an abnormality monitoring large model, perform abnormality monitoring on the tested target, and obtain monitoring results;

[0074] Step 400, multi-model collaboration step: determine the execution order and parameter configuration of each model according to the overall test goal, and issue task instructions to perform fuzz testing of the target software service; dynamically adjust the fuzz testing strategy based on the output of the target monitoring and feedback large model; summarize the output results of each model, perform data fusion and analysis to form the final test report.

[0075] In step 100, the industrial software network analysis step specifically includes:

[0076] A large model for industrial software network analysis is constructed, and formalized protocol elements are obtained from the protocol source code dataset, the protocol standard document dataset, and the protocol traffic dataset for learning. The semantic constraints between protocol elements are inferred, and diversified input data is generated based on prompt words to cover different paths and boundary conditions of the protocol, thereby generating the optimal fuzz testing seed samples.

[0077] In step 400, the multi-model collaboration step specifically includes:

[0078] Task scheduling: split the fuzz testing target into subtasks that can be executed by each model; configure parameters for each subtask; send each subtask together with the configuration parameters to the corresponding model for execution;

[0079] Result integration and process control: Summarize the output results of each model and provide feedback on the test information; perform correlation analysis and failure mode analysis on the execution results of each model to determine whether potential vulnerabilities are found; and monitor the execution status of the entire test process and intervene and adjust when abnormal situations are encountered.

[0080] Test strategy optimization: timely adjust the seed scheduling strategy according to the feedback information in the test, screen the seeds from the seed queue, and select more suitable seed samples; adjust the probability of the mutation strategy according to the coverage information to cover more boundary conditions and improve the coverage of fuzz testing; filter the seeds according to the results of the test feedback, and select the seeds that trigger more anomalies to add to the seed queue.

[0081] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a program code for executing the method shown in the flowchart.

[0082] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0083] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0085] As another aspect, the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the multi-model collaboration method for fuzz testing of industrial software protocols described in the above embodiment.

[0086] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the multi-model collaboration method for fuzz testing of industrial software protocols described in the above embodiment.

[0087] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0088] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0089] It should be noted that in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "setting" and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances; the drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations.

[0090] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An industrial software protocol fuzz testing multi-model collaboration system, characterized in that: include: Industrial software network protocol analysis model, used to obtain the semantic information of industrial software protocols and generate fuzz test seed queues in combination with prompt words; The test sample generation model is used to establish mutation strategies according to the constraints in the target software fuzz test, and mutate the fuzz test seeds through knowledge reasoning to generate test samples; The target monitoring and feedback model is used to perform fuzz testing on the target software service according to the test samples, monitor the abnormalities of the tested target software service, and feed back the monitoring results to the multi-model collaboration agent; Multi-model collaboration agent, used to determine the execution order and parameter configuration of each model according to the overall test goal, dynamically adjust the fuzzy test strategy based on the output of the target monitoring and feedback large model, monitor the execution status of the entire test process and intervene and adjust when encountering abnormal situations, and summarize the output results of each model and perform data fusion and analysis to form the final test report; The working process of the industrial software network protocol analysis large model includes: Obtaining a protocol source code dataset, a protocol standard document dataset, and a protocol traffic dataset; Filter the protocol source code data to obtain relevant codes, split the codes to obtain code blocks, and finally extract key information from the code blocks as feature vectors; Split and block the protocol standard document data set, obtain standard text blocks and protocol paradigms, and input them into the industrial software network protocol analysis model; For the protocol traffic data set, the packets are segmented based on the session granularity to remove the interference information in the session. The protocol session packet sequence is provided as input to the industrial software network protocol analysis model, which extracts the protocol information from the session sequence. The industrial software network protocol analysis model learns from formalized protocol elements including protocol feature vectors, protocol paradigms, and session sequences, infers semantic constraints between protocol elements, and generates diverse input data based on prompt words to cover different paths and boundary conditions of the protocol, generating the optimal fuzz test seed sequence. The strategy of dynamically adjusting fuzzy testing in the multi-model collaborative agent includes: timely adjusting the seed scheduling strategy according to the feedback information in the test, screening the seeds from the seed queue, and selecting seed samples that better meet the requirements; adjusting the probability of the mutation strategy according to the coverage information to cover more boundary conditions; filtering the seeds according to the results of the test feedback, and selecting the seeds that trigger more anomalies to add to the seed queue.

2. The industrial software protocol fuzz testing multi-model collaboration system according to claim 1 is characterized in that: In the test sample generation large model, a single strategy or a combination of multiple strategies is selected to mutate the seed, wherein the mutation strategies include bit / byte operation, number mutation, string mutation, checkpoint bypass and injection attack.

3. The industrial software protocol fuzz testing multi-model collaboration system according to claim 1 is characterized in that: The target monitoring and feedback model inputs the generated test cases into the target system, performs status tracking, coverage statistics and anomaly monitoring on the test process, captures various system events during the execution process in real time, and outputs the test results.

4. The industrial software protocol fuzz testing multi-model collaboration system according to claim 1 is characterized in that: The working process of the multi-model collaborative agent includes: task decomposition: splitting the fuzzy test target into subtasks that can be executed by each model; task configuration: configuring parameters for each subtask; task distribution: distributing each subtask together with the configuration parameters to the corresponding model for execution.

5. The industrial software protocol fuzz testing multi-model collaboration system according to claim 1 is characterized in that: The working process of the multi-model collaboration agent also includes: summarizing the output results of each model and feeding back test information; and performing correlation analysis and failure mode analysis on the execution results of each model to determine whether potential vulnerabilities are found.

6. A multi-model collaboration method for industrial software protocol fuzz testing, implemented based on the multi-model collaboration system for industrial software protocol fuzz testing according to any one of claims 1 to 5, characterized in that: include: Industrial software network analysis steps: By building a large industrial software network analysis model, formalized protocol elements are obtained from the protocol source code data set, protocol standard document data set, and protocol traffic data set for learning, and the semantic constraints between protocol elements are inferred. Diverse input data is generated based on prompt words to cover different paths and boundary conditions of the protocol, and the optimal fuzz test seed samples are generated; Test case generation step: Generate a large model by constructing test samples, and establish a mutation strategy based on the constraints in the target software fuzz test. Mutate the fuzz test seed through knowledge reasoning of the large model generated by the test samples to generate test samples; Abnormal monitoring step: by building a large model of target monitoring and feedback, fuzz testing is performed on the target software service according to the test samples, and abnormal monitoring is performed on the tested target software service at the same time, and the monitoring results are fed back; Multi-model collaboration steps: Determine the execution order and parameter configuration of each model according to the overall test objectives, and issue task instructions to perform fuzz testing of the target software service; Dynamically adjust the fuzz testing strategy based on the output of the target monitoring and feedback model; Summarize the output results of each model, perform data fusion and analysis to form the final test report; The multi-model collaboration step specifically includes: Task scheduling: split the fuzz testing target into subtasks that can be executed by each model; configure parameters for each subtask; send each subtask together with the configuration parameters to the corresponding model for execution; Result integration and process control: Summarize the output results of each model and provide feedback on the test information; perform correlation analysis and failure mode analysis on the execution results of each model to determine whether potential vulnerabilities are found; and monitor the execution status of the entire test process and intervene and adjust when abnormal situations are encountered; Test strategy optimization: timely adjust the seed scheduling strategy according to the feedback information in the test, screen the seeds from the seed queue, and select seed samples that better meet the requirements; adjust the probability of the mutation strategy according to the coverage information to cover more boundary conditions; filter the seeds according to the results of the test feedback, and select seeds that trigger more anomalies to add to the seed queue.

Images