A method for generating fuzzy test cases for industrial control protocols based on Transformer decoder

Abstract

This invention discloses a method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder, belonging to the field of industrial network security. The method includes: 1) obtaining a test case training set; 2) training set preprocessing, preprocessing the training set into sequence data and performing format processing to conform to the input of the Transformer decoder model; 3) model training, obtaining a learning model that conforms to the protocol format specifications through training; 4) test case generation; 5) mutation operator replacement; and 6) feedback optimization, monitoring the initial fuzzing response, constructing a manual training set based on abnormal test case clustering, and retraining. This invention efficiently learns industrial control protocol formats based on the Transformer decoder model, and combines interval selection strategies and key segment weighting algorithms to generate effective, diverse, and targeted test cases, improving the efficiency of fuzzing.

Description

Technical Field

[0001] This invention relates to a method for generating test cases for industrial protocols, and more particularly to a method for generating fuzzy test cases for industrial control protocols based on Transformerdecoder. Background Technology

[0002] With the development of industrial informatization, Industrial Control Systems (ICS) are gradually integrating with information technology and the Internet, realizing factory intelligence and improving production efficiency. Industrial Control Protocols (ICPs), as the bridge for communication between various parts of an information system, are crucial to the security of ICS. However, the initial design of ICPs lacked consideration for protocol security, making inherent security issues increasingly prominent. Attackers can launch attacks such as information injection, information leakage, and denial-of-service attacks through specific data packets, causing serious security risks. In response to the increasingly serious industrial control security problems, proactive vulnerability discovery and patching of ICPs can help improve the security and reliability of industrial control systems.

[0003] Fuzz testing is currently an effective technique for discovering vulnerabilities in industrial control system (ICS) protocols. It involves sending a large number of unexpected test cases to the target system and monitoring the system's response to determine if a vulnerability has been found. Based on the method of test case generation, it can be divided into mutation-based methods and generation-based methods. Mutation-based methods suffer from the defect of blind mutation, leading to a large number of invalid test cases. Therefore, most current methods utilize generation-based methods to generate test cases. However, the widespread use of proprietary protocols makes it difficult for researchers to obtain effective protocol specification information, hindering the generation conditions of generation-based methods. Therefore, there is an urgent need for a fuzz test case generation method for unknown ICS protocols.

[0004] The achievements of deep learning in Natural Language Processing (NLP) have provided new insights into the learning of unknown industrial control (ICP) protocol format specifications. In 2017, the Google Brain team proposed the Transformer neural network architecture. Through character embedding and a multi-layer encoder-decoder structure, Transformer can generate continuous and accurate text based on pre-training on unlabeled text data, achieving significant results in tasks such as text generation, machine translation, and intelligent dialogue. Industrial protocols are typically represented as hexadecimal character sequences, exhibiting characteristics similar to natural language. Therefore, the problem of learning unknown ICP protocol format specifications can be transformed into a problem of predicting text sequence data. Leveraging the advantages of this model, the learning and parsing of ICP protocols can be effectively optimized and extended, improving the understanding and processing capabilities of protocol formats. Summary of the Invention

[0005] The purpose of this invention is to provide a method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder. This method uses a Transformer decoder model to learn the format specifications of industrial control protocols, utilizes a multi-head attention mechanism to focus on information in different subspaces of the data, effectively extracts data features, and combines interval selection strategies, byte importance algorithms, and a mutation operator library for data prediction and generation. Compared with existing methods, the proposed method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder is superior in terms of test case acceptance rate, diversity, and relevance.

[0006] This invention is achieved through the following technical solution:

[0007] A method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder, the method comprising the following steps:

[0008] S1. Obtain test case training set: Capture the communication traffic of the industrial scenario simulation platform and simulation software environment, analyze the data packet interaction content, and select request data packets as training set;

[0009] The specific steps of step S1 are as follows:

[0010] S101. Analyze the captured communication data traffic, determine the response relationship between communication objects, extract the Query message sent by the host computer, and save it as a traffic packet containing only request data.

[0011] S102. Repeat step S101 until a considerable amount of diverse request data traffic is collected.

[0012] S2. Data preprocessing: The request data packet content is processed into hexadecimal sequence data, classified by multiple clustering methods, and each data category is divided by bytes by adding spaces. The length of each type of sample is uniform to conform to the input format of the Transformerdecoder model.

[0013] The specific steps of step S2 are as follows:

[0014] S201. Process the request message data packets filtered in step S1 into sequence data in hexadecimal character format;

[0015] S202. Divide the test case samples into bytes, with the byte value ranging from 0 to 255. Separate adjacent bytes with spaces to meet the requirements of the model input data format.

[0016] S203. Combine multiple clustering strategies to classify the samples and determine whether the types of test cases are sufficient for model training. If the number is insufficient, data augmentation techniques are needed to generate enough test samples.

[0017] S204. Standardize the test samples and unify the length of all test samples. Assuming the standard sample length is set to t, truncate samples with more than t bytes to the standard length, and add a 256 padding flag to the end of test samples with less than t bytes to make up the length.

[0018] S3. Model Training: Using the data processed in step S2 as the model input, the model is trained through a teacher-forced strategy to enable it to fully learn the inherent format specification information of the protocol and obtain a test case generation model based on the Transformer decoder.

[0019] S4. Generate test cases, customize the initial input data content and length, and the model generates data using autoregressive methods; adaptively adjust the output probability interval for each time step through an interval selection strategy.

[0020] S5, Mutation Operator Replacement: Collect mutation operators that may cause anomalies from publicly disclosed vulnerability data, build a mutation operator library, calculate the weight of each byte using a byte importance algorithm, replace the corresponding type of mutation operator with the high-weight part, build a set of manual test cases, and use them for training and fuzzing.

[0021] S6. Feedback and optimization: Perform fuzz testing and monitor response information, mark abnormal test cases and cluster them, use the sample set containing marked test cases as a new training set for retraining, repeat the whole process, and continuously update the generated test cases.

[0022] The Transformer decoder learning model consists of 12 stacked decoder layers. Each decoder layer comprises an embedding layer, a causal attention layer, a normalization layer, a feedforward network, a residual network, and a softmax layer. The embedding layer is responsible for embedding the input data into a high-dimensional vector of 768 dimensions and adding absolute position encoding to give the data order information. The causal attention layer uses the causal attention mechanism to extract data features and creates a mask matrix to assist in teacher-forced policy training. The feedforward network is expanded to a dimension of 2048 to enhance the model's expressive power. The softmax layer determines the final output content selection.

[0023] The method for generating fuzzy test cases for industrial control protocols based on Transformer decoder describes a method where the industrial control protocol data packets in S1 are obtained from various PLC devices from different manufacturers in the actual working conditions of the oil and gas gathering and transportation industrial simulation platform in the Key Laboratory of Information Security in the Petrochemical Industry. These packets are captured and collected using Wireshark sniffing software. Simultaneously, communication data packets are collected from GitHub projects as subsequent training and test sets. Commonly used industrial control system communication protocols include Modbus TCP, S7comm, Ethernet / IP, and DNP3.

[0024] The method for generating fuzzy test cases for industrial control protocols based on Transformer decoder, in step S203, analyzes whether the diversity of training data is satisfied by performing multiple clustering methods on the test cases. If not, sufficient training data is generated through other data augmentation methods.

[0025] The aforementioned method for generating fuzzy test cases for industrial control protocols based on Transformer, wherein S3 uses a Transformer decoder model to learn the industrial control protocol format specifications; it captures features of different subspaces through a causal attention mechanism, effectively handles sequence data exceeding general length, captures long-distance dependencies between bytes, better copes with the format learning of unknown protocols and complex real-world working environments, and improves the learning efficiency of industrial control protocol format specifications.

[0026] The method for generating fuzzy test cases for industrial control protocols based on Transformer decoder describes a method where the content and length of initial data are freely defined in S4 to guide subsequent autoregressive test case generation. The interval selection strategy aims to dynamically adjust the probability selection interval of each time step during the generation process, focusing on the selection of key field content to improve the diversity and relevance of test cases.

[0027] The method for generating fuzzy test cases for industrial control protocols based on Transformer decoder, as described in S5, proposes to construct a mutation operator library, calculate the weight of each byte of the test case using a byte importance algorithm, and attempt to increase the frequency of replacing corresponding mutation operators with high-weight fields, thereby constructing a set of manual test cases, which are used for both fuzz testing and model retraining, and are repeated.

[0028] The method for generating fuzzy test cases for industrial control protocols based on Transformer decoder includes the following steps: In step S6, fuzzy test response information is monitored, abnormal test cases are marked, and clustering is performed using an improved k-means method. The sample set containing marked test cases is then incorporated into the training set for retraining. The entire process is repeated to continuously update the generated test cases.

[0029] The aforementioned industrial control protocol fuzzy test case generation system based on Transformer decoder includes a training data acquisition module: used to acquire communication data between the industrial simulation platform and the simulation software environment, and to filter request data;

[0030] Data preprocessing module: used to process request data packets into hexadecimal sequence form, and to perform data augmentation on minority class samples through various clustering strategies until a predetermined number of samples are collected as a training set;

[0031] Model training module: Implemented based on the Transformer decoder model, using the processed dataset as the training set for model training;

[0032] Test generation module: Combines interval selection strategy to generate autoregressive test cases as test sets;

[0033] Feedback and optimization module: Uses the test set to perform fuzz testing, monitors the response information of the device under test, marks abnormal test cases and clusters them, replaces the sample set containing marked test cases with the byte importance algorithm-assisted mutation operator, constructs a manual test case set, and is used for fuzz testing and model retraining.

[0034] The advantages and effects of this invention are:

[0035] This invention utilizes a Transformer decoder to directly learn unknown industrial control protocol format specifications. The multi-head attention mechanism not only captures subspace information of different elements in data packets and effectively extracts data features, but also significantly improves model training speed through parallel processing. Furthermore, this method can accept data of arbitrary length and content for data generation, enhancing compatibility with unknown protocols. The interval selection strategy supports adaptive dynamic adjustment of the probability selection interval of output data at specific time steps, increasing the diversity of test cases. Monitoring the response of the target under test and combining it with a key field localization algorithm to perform ordinary mutation or replacement mutation operators on high-weight fields, a manual test case set is constructed. This set is used for fuzz testing and model retraining, guiding the model to generate more test cases with a bias towards anomalous formats. This invention solves the problems of low test case acceptance rate, weak targeting, and poor diversity in traditional methods. In addition, this model has high adaptability and scalability, and is compatible with various industrial control protocols such as Modbus TCP, S7comm, Ethernet / IP, and DNP3, as well as other unknown industrial control protocols. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating a method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder, according to the present invention.

[0037] Figure 2 This is a structural diagram of the test case generation model based on Transformer decoder provided by the present invention;

[0038] Figure 3 This is a structural diagram of a system for generating fuzzy test cases for industrial control protocols based on a Transformer decoder, provided by the present invention.

[0039] Figure 4 The average acceptance rate of test cases generated for the Modbus TCP protocol based on the Transformer decoder test case generation model provided by this invention patent is experimentally measured on three simulators. Detailed Implementation

[0040] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] The process of this invention is as follows Figure 1 As shown, the specific steps are as follows:

[0042] S1. Obtain test case training set: Capture the communication traffic of the industrial scenario simulation platform and simulation software environment, analyze the data packet interaction content, filter request data packets according to IP address or port and save them as training set;

[0043] S2. Data preprocessing: The requested dataset is processed into hexadecimal sequence data. Clustering is performed using various clustering methods such as frame length clustering. Data of each class is divided into bytes, with bytes separated by spaces. Each sample is standardized to a uniform length and padded with 256 characters to make the samples conform to the input format of the Transformerdecoder model.

[0044] S3. Model Training: Using the data processed in step S2 as the model input data, the model is trained through teacher-forced strategies and masking mechanisms to fully explore the deep-level format specifications of the protocol and obtain a test case generation model based on Transformer decoder.

[0045] S4. Generate test cases, customize the initial input data content and length, and the model will generate subsequent data through autoregression; the probability selection interval for a specific time step will be adaptively and dynamically adjusted through the interval selection strategy.

[0046] S5. Mutation Operator Replacement: Collect mutation operators that may cause system anomalies from publicly disclosed vulnerability data, build a mutation operator library, calculate the weight of each byte through the byte importance algorithm, replace the corresponding type of mutation operator with the high weight byte or perform ordinary mutation, such as bit flipping, random addition or subtraction of 1, thereby build a set of manual test cases, which are also used for model retraining and fuzz testing.

[0047] S6. Feedback and Optimization: Monitor the response information of the fuzzing process. If an anomaly is triggered, mark the abnormal test cases. Replace the original k-means clustering method that relies on Euclidean distance with Smith-Waterman clustering. Cluster the test cases using the improved k-means clustering method. Use the sample set containing the marked test cases as a new training set and integrate it into the original training set for model retraining. Repeat the entire process and continuously update the generated test cases.

[0048] In one embodiment of the present invention, the specific steps of step S1 are as follows:

[0049] S101. Analyze the captured communication data packets to determine the response relationship between communication entities, extract the Query packets sent by the host computer based on the IP address or port number, and save them as traffic packets containing only request data.

[0050] S102. Repeat step S101 until the expected amount of request data traffic with sufficient functionality is collected.

[0051] In one embodiment of the present invention, step S2 specifically involves the following steps:

[0052] S201. Output the request message data packets filtered in step S1 as sequence data in hexadecimal format;

[0053] S202. Divide the test case samples into bytes, with each byte having a value range of 0-255, and adjacent bytes separated by spaces to meet the model input format requirements.

[0054] S203. Combine multiple clustering strategies to classify the samples and determine whether the types of test cases are sufficient. If not, data augmentation methods should be combined to generate enough test samples for the minority classes.

[0055] S204. Standardize the test cases, that is, unify the length of all test samples. Assuming the standard sample length is set to t, for samples with more than t bytes, truncate them to the standard length, and for samples with less than t bytes, add a 256 flag at the end to pad them to the standard length.

[0056] The protocol format specification learning model used in step S3 is the Transformer decoder learning model:

[0057] like Figure 2 As shown, the Transformer decoder learning model consists of 12 stacked decoder layers. Each decoder layer comprises a causal attention layer, a normalization layer, a feedforward network, a residual network, and a softmax layer. The embedding layer is responsible for converting the input data into embedding vectors and adding absolute position encoding to impart data order information. The causal attention layer uses a causal attention mechanism to extract data features and creates a mask matrix to assist in teacher-forced policy training. The fully connected layer has 768 neurons and plays an important role in each part of the model, mainly used to integrate the extracted data features. The feedforward network is expanded to a dimension of 2048 to enhance the model's expressive power. The softmax layer outputs the probability distribution on the vocabulary.

[0058] Before entering the embedding layer, the serialized test case samples are first segmented into smaller units, and a vocabulary is constructed using the BEP (BytePairEncoding) algorithm, assigning each sub-word a unique Token ID. A start marker

[257] is added to the beginning of the sample, and an end marker

[258] is added to the end; a padding marker

[256] is added to complete the length to the standard length. The embedding layer includes Token embedding and Position embedding. Token embedding maps each Token ID to a vector representation of [1, 768] dimensions, while Position embedding assigns a unique position to each Token, compensating for the original model's lack of encoding of input order, and mapping it to an embedding vector ([1, 768]). After the above processing, each sequence is represented as a tensor of [1, t, 768] (where t represents the sample length).

[0059] This model primarily utilizes the decoder portion of the Transformer, consisting of 12 stacked decoder layers, each with 12 causal attention blocks. This model effectively learns the format specifications of unknown industrial control protocols, and compared to traditional models, the Transformer can effectively capture long-range dependencies between sample data.

[0060] The powerful learning ability of the Transformer decoder relies on a multi-head causal attention mechanism. Compared with the single-head attention mechanism, the multi-head attention mechanism effectively extracts information from different subspaces of the data. Its basic calculation process is as follows:

[0061] The embedded vector and multiple different weight matrices W i Q W i K W i V Calculations yielded multiple sets of Q. i ,K i V i Each head pair matrix Q i ,K i The matrix is ​​used to calculate the corresponding attention score using equation (1):

[0062]

[0063] Attention scores are normalized using the softmax function to obtain the attention distribution β. ih As shown in equation (2):

[0064]

[0065] With the corresponding V hThe output of each attention head is calculated, and the outputs of each head are concatenated with the transformation matrix W. O The final predicted output O is calculated, as shown in equation (3), where

[0066]

[0067] During model training, cross-entropy is chosen as the loss function to evaluate the model's prediction accuracy. This function can effectively reduce the risk of gradient vanishing during stochastic gradient descent. The Adam (Adaptive Moment Estimation) optimizer is selected to optimize the loss function.

[0068] During the output process, a range selection strategy is used to precisely control the probability selection range of a specific time step. A random threshold between 0 and 1 is set. If the maximum output probability of a specified time step is greater than the threshold, a random selection is made from N minimum probability values. If the maximum probability value of the time step is less than the random threshold, a random selection is made from N maximum probability values ​​as the output. To eliminate the risk of selecting the end marker as the output at a critical time step, a dynamic filtering mechanism is set up. Based on the learned knowledge, the mechanism automatically adjusts when to start and when to stop the filtering steps to ensure the normal generation of test cases.

[0069] Collect vulnerability information from public vulnerability databases, summarize mutation operators, construct mutation operator library, use keyword byte positioning algorithm as shown in Equation (4), combine fuzz testing process to calculate the weight w'(i,k) of each field, and perform common mutation operations such as bit flipping, addition, subtraction, insertion, deletion, swapping, random combination, etc. or replace the corresponding mutation operator on the high weight part;

[0070] Byte importance is calculated as shown in equation (4):

[0071]

[0072] Where, N i R(i,k) represents the number of test cases sent to the i-th function code in the system under test, and D(i,k) represents the number of times this position caused an exception in the test in history and the number of test cases since the last mutation, respectively.

[0073] Monitor the response of the system under test. If an anomaly is triggered, mark the test case. Use an improved clustering algorithm to find a set of test cases with a certain degree of similarity to the anomaly test case. Update the set using the above mutation operation to build a new set of manual test cases, which is also used for fuzz testing and model retraining.

[0074] like Figure 3The present invention also includes a system for generating fuzzy test cases for industrial control protocols based on a Transformer decoder, as shown in the structural diagram, characterized by comprising the following parts:

[0075] The training data acquisition module is used to acquire communication traffic in actual industrial scenarios as model training data. Currently, it is mainly acquired through three methods: (1) collecting actual factory communication data traffic and capturing it using Wireshark software; (2) capturing communication traffic by performing various executable operations on the laboratory oil and gas gathering and transportation simulation platform and capturing communication data using Wireshark software; (3) finding publicly available datasets on the Internet, such as the Github platform.

[0076] The data preprocessing module is used to preprocess the dataset to conform to the data input format of the Transformer decoder model;

[0077] The model training module is implemented based on the Transformer decoder model and built on the PyTorch framework. It calls the model through the Hugging face interface and trains it using the processed dataset mentioned above.

[0078] The test module is generated by using an interval selection strategy to assist the trained model in generating test cases.

[0079] The feedback and optimization module monitors fuzz test results, builds a mutation operator library, and dynamically adjusts the mutation method of key fields through a byte importance localization algorithm; manual test cases are used for both model retraining and fuzz testing.

[0080] The following experiment verifies the effectiveness of the industrial control protocol fuzz test case generation method based on Transformer decoder in fuzz testing of Modbus TCP protocol.

[0081] The method provided in this invention tests the common Modbus TCP protocol. The training data comes from the following parts: (1) a simulation environment conforming to the Modbus TCP protocol is built using Modbus Slave, ModRSSim, and Modbus Sim32 to capture communication traffic; (2) communication data of the laboratory oil and gas gathering and transportation simulation platform is captured; (3) publicly available network data is collected. The first two parts use Wireshark sniffing software to capture traffic. The above traffic data is preprocessed, and data augmentation is performed on minority class samples to meet the requirements of model training. The specific experimental process is as follows:

[0082] Experimental environment: The test computer is Core TMi7-12700H CPU @ 2.7GHz, 8GB RAM, Windows 10 Professional; Development language: Python 3.9; Development environment: PyTorch; Switch model: H3C-S5120V2-2SP-SI; Simulated slaves: Modbus Slave v6.2.2, ModRssim v8.2.0, and Modbus Sim32v4.0; Rockwell AB1769L-30ER;

[0083] Experimental Procedure: Preprocessed data was input into the model in batches for training. Based on the training loss curve, it was determined when to start saving the model after different training epochs at the same training interval for subsequent generation. Combining an interval selection strategy, the numerical selection interval at each time step was dynamically controlled to increase the diversity of test cases. After generating a certain number of test cases, a script written in Python was used to send test cases to three simulation slave stations and receive the slave stations' responses to each test case. After sending, abnormal test cases were marked, and the clusters were re-clustered using an improved k-means clustering algorithm. The clusters containing abnormal test cases were mutated using a byte importance algorithm and a mutation operator library to construct a new dataset, which was used for fuzz testing and model retraining.

[0084] Experimental Description: To obtain high-quality training data, after extracting the payload portion of the Modbus TCP protocol, the data needs to be clustered to determine the quantity of data in each category. Since data frames of the same length usually belong to the same function, frame length clustering is used to help determine the quantity of each category. Data augmentation techniques are used to augment the data for minority classes. To adapt to model requirements, the length of the training data needs to be standardized. The key part of this protocol, i.e., the data portion, is located at the end of the data sequence. To avoid losing key information, the average sequence length after clustering is used as the standard length. Data frames shorter than this length are padded to the standard length using a 256 padding flag. In addition, based on the loss curve of the model training, it can be inferred that the loss tends to stabilize after 30 epochs. At this point, the model is saved every 5 epochs. Using different stages of learning models to generate test cases can effectively improve the diversity of test cases based on different learning effects. The model parameters are designed as follows: embedding dimension is 768, feedforward layer expansion dimension is 2048, number of decoders is 12, each decoder has 8 attention heads, activation function is GELU (Gaussian Error Linear Unit), optimizer is Adam (Adaptive Moment Estimation), and training period is 50 epochs.

[0085] Experimental Results Analysis: Experiments were conducted on Modbus Slave, ModRSSim, and Modbus Sim32 simulated slave stations. The test case acceptance rate, vulnerability detection rate, and types of anomalies for the three slave stations were statistically analyzed. Figure 4 As shown, the test cases generated by this invention achieved an average acceptance rate of 88.2% across three simulated slave stations, an average vulnerability detection rate of 3.7%, and an average of five anomaly types detected. Compared to general fuzzy tools Peach and Sully, this represents an increase in acceptance rate of 63.5% and 32.2%, respectively, and an increase in vulnerability detection rate of 2.3% and 1.6%, respectively.

[0086] Experimental results show that, under the same conditions, the fuzz test cases generated by this invention outperform existing fuzzing tools in terms of acceptance rate, vulnerability discovery capability, and vulnerability discovery types, thus improving the efficiency of vulnerability discovery.

[0087] After the entire model method was completed, it was tested on a fuzzing benchmark platform. Compared with general fuzzing tools such as Sully and Peach, this method generates more targeted test cases and shows significant advantages in terms of efficiency, diversity, and vulnerability discovery capabilities.

[0088] This invention employs the above technical solution. In the process of fuzz testing of industrial control protocols, it uses data captured by an industrial simulation platform, simulation software data, and publicly available online datasets as training data. For minority class samples, data augmentation techniques are used to expand the data. By training the Transformer decoder model, the protocol format specification features can be automatically learned. After training, the model can execute subsequent test case generation tasks. Combined with interval selection strategies, byte importance localization algorithms, and mutation operator libraries, more targeted and diverse test cases are generated, thereby improving the efficiency and universality of industrial control protocol vulnerability discovery. The beneficial effects of this invention are as follows: 1) It effectively extracts high-dimensional features of industrial control protocol data by using word embedding, positional encoding, and multi-head causal attention mechanisms of the Transformer decoder model, making it suitable for unknown protocol sequences far exceeding standard lengths; 2) It effectively increases the diversity of generated test cases by utilizing interval selection strategies; 3) It enhances the relevance of test cases by combining mutation operator libraries and byte importance localization algorithms, making the entire fuzzing process increasingly tend to generate biased and anomalous test cases, thus improving the effectiveness of vulnerability discovery; 4) Through comparison with general fuzzing tools on a fuzzing benchmark platform, it is shown that the industrial control protocol fuzzing test case generation model based on Transformer decoder can ensure the effectiveness of test cases while maintaining their diversity and relevance, thereby improving the overall efficiency of industrial control protocol vulnerability discovery.

[0089] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder, characterized in that, The method includes the following steps: S1. Obtain test case training set: Capture the communication traffic of the industrial scenario simulation platform and simulation software environment, analyze the data packet interaction content, and select request data packets as training set; The specific steps of step S1 are as follows: S101. Analyze the captured communication data traffic, determine the response relationship between communication objects, extract the Query message sent by the host computer, and save it as a traffic packet containing only request data. S102. Repeat step S101 until the expected amount of diverse request data traffic is collected. S2. Data preprocessing: The request data packet content is processed into hexadecimal sequence data, classified by multiple clustering methods, and each data category is divided into bytes by adding spaces. The length of each type of sample is uniform to conform to the input format of the Transformer decoder model. The specific steps of step S2 are as follows: S201. Process the request message data packets filtered in step S1 into sequence data in hexadecimal character format; S202. Divide the test case samples into bytes, with the byte value ranging from 0 to 255. Separate adjacent bytes with spaces to meet the requirements of the model input data format. S203. Combine multiple clustering strategies to classify the samples, determine whether the types of test cases are sufficient for model training, and if the number is insufficient, generate enough test samples through data augmentation techniques. S204. Standardize the test samples and unify the length of all test samples. Assuming the standard sample length is t, truncate samples with more than t bytes to the standard length, and add a 256 padding flag to the end of test samples with less than t bytes. S3. Model Training: Using the data processed in step S2 as the model input, the model is trained through a teacher-forced strategy to enable it to fully learn the inherent format specification information of the protocol and obtain a test case generation model based on the Transformer decoder. S4. Generate test cases, customize the initial input data content and length, and the model generates data using autoregressive methods; adaptively adjust the output probability interval for each time step through an interval selection strategy. S5, Mutation Operator Replacement: Collect mutation operators that may cause anomalies from publicly disclosed vulnerability data, build a mutation operator library, calculate the weight of each byte using a byte importance algorithm, replace the corresponding type of mutation operator with the high-weight part, build a set of manual test cases, and use them for training and fuzzing. S6. Feedback and optimization: Perform fuzz testing and monitor response information, mark abnormal test cases and cluster them, use the sample set containing marked test cases as a new training set for retraining, repeat the whole process, and continuously update the generated test cases. The Transformer decoder learning model consists of 12 stacked decoder layers. Each decoder layer comprises an embedding layer, a causal attention layer, a normalization layer, a feedforward network, a residual network, and a softmax layer. The embedding layer is responsible for embedding the input data into a high-dimensional vector of 768 dimensions and adding absolute position encoding to give the data order information. The causal attention layer uses the causal attention mechanism to extract data features and creates a mask matrix to assist in teacher-forced policy training. The feedforward network is expanded to a dimension of 2048, and the softmax layer determines the final output content selection.

2. The method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder according to claim 1, characterized in that, The industrial control protocol data packets in S1 are sourced from various PLC devices from different manufacturers in the actual working conditions of the oil and gas gathering and transportation industrial simulation platform in the Key Laboratory of Information Security in the Petrochemical Industry. They are captured and collected using Wireshark sniffing software. At the same time, communication data packets are collected from GitHub projects as subsequent training and testing sets. The industrial control system communication protocols include Modbus TCP, S7comm, Ethernet / IP, and DNP3.

3. The method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder according to claim 1, characterized in that, In step S203, the diversity of the training data is analyzed by performing various clustering methods on the test cases. If the diversity is not satisfied, sufficient training data is generated through other data augmentation methods.

4. The method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder according to claim 1, characterized in that, The S3 described above uses a Transformer decoder model to learn the protocol format specification; it captures features of different subspaces through a causal attention mechanism, effectively handles sequence data exceeding the general length, and captures long-distance dependencies between bytes.

5. The method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder according to claim 1, characterized in that, In S4, the content and length of the initial data are freely defined to guide the subsequent autoregressive generation of test cases. The interval selection strategy adaptively and dynamically adjusts the probability selection interval for each time step during the generation process.

6. The method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder according to claim 1, characterized in that, The S5 section proposes to construct a mutation operator library, calculate the weight of each byte in the test case using a byte importance algorithm, and attempt to increase the frequency of replacing the corresponding type of mutation operator in high-weight fields, construct a set of manual test cases, and use them for fuzz testing and model retraining, repeating the process.

7. The method for generating fuzzy test cases for industrial control protocols based on a Transformer decoder according to claim 1, characterized in that, In step S6, the fuzz test response information is monitored, abnormal test cases are marked, and clustering is performed using the improved k-means method. The sample set containing marked test cases is then incorporated into the training set for retraining. This process is repeated to continuously update the generated test cases.

8. A fuzzy test case generation system for industrial control protocols based on a Transformer decoder, used in the method of any one of claims 1 to 7, characterized in that: Training data acquisition module: used to acquire communication data between the industrial simulation platform and the simulation software environment, and to filter requested data; Data preprocessing module: used to process request data packets into hexadecimal sequence form, and to perform data augmentation on minority class samples through various clustering strategies until a predetermined number of samples are collected as a training set; Model training module: Implemented based on the Transformer decoder model, using the processed dataset as the training set for model training; Test generation module: Combines interval selection strategy to generate autoregressive test cases as test sets; Feedback and optimization module: Uses the test set to perform fuzz testing, monitors the response information of the device under test, marks abnormal test cases and clusters them, replaces the sample set containing marked test cases with the byte importance algorithm-assisted mutation operator, constructs a manual test case set, and is used for fuzz testing and model retraining.

Images