A Modbus TCP protocol test case generation method based on generative adversarial networks

Abstract

A Modbus TCP protocol test case generation method based on a generative adversarial network (GAN) includes the following steps: S1: extracting target protocol message data from captured network protocol messages and preprocessing this data as a model training dataset; S2: constructing an efficient training model by combining a GAN, a bidirectional long short-term memory (LSTM) network, and a self-attention mechanism, and training the model using the training dataset; S3: determining whether the training model end condition is met. If so, saving the generator model parameters and generating test cases based on the parameters; otherwise, skipping to S2; S4: mutating the test case using a mutation strategy from a genetic algorithm; S5: sending the test case to the PLC under test via a socket, monitoring the status of the tested system, and obtaining a response code. This method addresses the issues of incomplete format learning, low response rate, and high redundancy in Modbus TCP protocol test case generation, thereby improving fuzz testing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of fuzzy technology, and in particular to a method for generating Modbus TCP protocol test cases based on a generative adversarial network. Background Art

[0002] With the continuous evolution of the Industrial Internet and the accelerating integration of informatization and industrialization, the boundaries between traditional industrial control systems and the Internet are becoming increasingly blurred. Industrial control systems, originally designed for intranet communication, are increasingly being connected to the Internet. While this integration has greatly improved production efficiency and openness, it has also exposed traditional industrial control systems to increasingly severe Internet security threats. Fuzz testing, at its core, involves injecting unexpected and anomalous input data into the target system and closely monitoring the system's response to identify and locate potential vulnerabilities or defects in software or devices. Protocol fuzz testing has become a highly targeted and effective testing method. This method uses carefully crafted malformed data packets to simulate malicious or anomalous network communication behavior. These packets are then sent to the industrial control device under test, while closely monitoring the device's operating status and responses. This approach can detect and expose potential security vulnerabilities in the industrial control device's protocol processing.

[0003] Existing solutions include the introduction of sequence-based generative adversarial network (SEQ-GAN) models in protocol fuzz testing to optimize test case generation and improve comprehensiveness and efficiency. The SEQ-GAN model combines the concepts of generative adversarial networks and reinforcement learning, making it particularly suitable for generating discrete sequence data. The model's core approach is to leverage the adversarial training mechanism of GANs to generate realistic sequence data, while also drawing on the policy gradient method of reinforcement learning to address the difficulty of gradient updates for discrete data. However, current fuzz testing also presents its own challenges. These limitations primarily include incomplete learning of protocol formats and features, low test case response rates, and high redundancy. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a Modbus TCP protocol test case generation method based on a generative adversarial network, which solves the problems of incomplete format learning, low response rate and high redundancy in Modbus TCP protocol test case generation and improves the efficiency of fuzzy testing.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: a Modbus TCP protocol test case generation method based on a generative adversarial network, comprising the following steps:

[0006] S1: Extract target protocol message data from the captured network protocol messages and pre-process the data as a model training dataset;

[0007] S2: By combining generative adversarial networks, bidirectional long short-term memory networks, and self-attention mechanisms, an efficient training model is constructed and trained using the training dataset.

[0008] S3: Determine whether the training model end conditions are met. If so, save the parameters of the generator model and generate test cases based on the parameters. Otherwise, jump to S2.

[0009] S4: Use the mutation strategy in the genetic algorithm to mutate the test case;

[0010] S5: Send the test case to the PLC under test through Socket, monitor the status of the tested system and obtain the response code.

[0011] Preferably, step S1 includes the following steps:

[0012] S101: Capture protocol data packets when the host computer and PLC are communicating normally;

[0013] S102: extracting Modbus TCP protocol data from the data packet;

[0014] S103: Convert the Modbus TCP protocol data into binary format;

[0015] S104: Align the data by adding '0' after the short data to align all the data;

[0016] S105: After all the data are aligned, one-hot encoding is used to convert the format to form a training data set.

[0017] Preferably, step S2 includes the following steps:

[0018] S201: Considering that Bi-LSTM can effectively capture long-term dependencies and contextual information in sequence data, thereby generating more accurate and diverse sequences, and the self-attention mechanism helps the model better handle long sequences and global dependencies, improving the quality and diversity of generated sequences; therefore, the Bi-LSTM layer, self-attention mechanism layer, and fully connected layer are used as the generator of the generative adversarial network, and the Bi-LSTM layer, fully connected layer, and Softmax layer are used as the discriminator of the generative adversarial network; set the model parameters, and the number of data selected at a time , learning rate , pre-training times , number of adversarial training , minimum error These model parameters are pre-set values;

[0019] S202: The training dataset is used as input for pre-training of the generator. After the generator training is completed, fake data is generated; the fake data and the training dataset are used as the dataset of the discriminator in a 1:1 ratio for pre-training of the discriminator.

[0020] S203: Conduct adversarial training. First, fix the discriminator and train the generator. The generator generates false data samples similar to the real data. The false data samples generated by the generator are input into the discriminator together with the real data. The discriminator calculates a probability value through the loss function. , indicating the possibility that the input sample is real data; according to the loss function of the generator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the generator are updated to improve its ability to generate sample data that is highly similar to the real data; finally, the generator is fixed and the discriminator is trained; the discriminator receives real data samples and false data samples generated by the generator at the same time, and calculates the probability value that the false data sample can be regarded as real data according to the loss function of the discriminator. The goal is to minimize this loss function. According to the loss function of the discriminator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the discriminator are updated to improve its discrimination ability; these two steps are repeated until the stopping condition is reached. The discriminator loss function is as follows:

[0021] ;

[0022] in is the probability value, n is the total number of data selected at a time, is the discriminator’s prediction for the i-th sample, is the i-th true label.

[0023] Preferably, in step S3, when the probability value calculated by the discriminator loss function is When the error is less than the minimum error Min-loss, the training ends and the generator model parameters are saved. Test cases are generated according to the parameters. Otherwise, the training continues. The termination conditions are as follows:

[0024] ;

[0025] in is the probability calculated by the discriminator loss function, is the minimum error.

[0026] Preferably, step S4 includes the following steps:

[0027] S401: Binary encode the initial test case;

[0028] S402: Perform different mutation operations based on the abnormal code of the test case. For example, a normal individual performs a right shift operation or an XOR operation. Abnormal code 1 will change the function code to the initial function code and then perform an XOR operation. Others will perform an XOR operation.

[0029] S403: Decode the test case after mutation.

[0030] Preferably, in step S5, the data packet returned by the tested system is used to observe the state of the tested system, and a response code is obtained using a parsing module.

[0031] The present invention provides a Modbus TCP protocol test case generation method based on a generative adversarial network, which uses a generative adversarial network to generate a Modbus TCP protocol test case generation method, thereby improving the effectiveness and efficiency of the test. By integrating Bi-LSTM and self-attention mechanisms into a generative adversarial network model, the present invention successfully solves the problem of incomplete protocol format and feature learning in traditional fuzz testing, enabling the generated test cases to more accurately simulate the actual communication behavior of the Modbus TCP protocol, thereby effectively exposing potential security vulnerabilities. In addition, the present invention introduces a mutation strategy of a genetic algorithm to perform intelligent mutation on test cases, significantly reducing the redundancy of test cases and ensuring the diversity and pertinence of test data. This optimization not only reduces the complexity of the test process, but also significantly improves the response rate of test cases. The Modbus TCP protocol test cases constructed by the present invention have a high degree of protocol compliance, low redundancy and high response rate, which greatly improves the efficiency of fuzz testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0033] Figure 1 This is a diagram of the test case generation system architecture for the Modbus TCP protocol provided by the present invention;

[0034] Figure 2 This is a flow chart of a test case generation method for the Modbus TCP protocol provided by the present invention;

[0035] Figure 3 This is a structural diagram of the training model provided by the present invention. DETAILED DESCRIPTION

[0036] This paper designs a Modbus TCP protocol test case generation method architecture based on generative adversarial networks, which is implemented in Python. The architecture diagram is as follows: Figure 1As shown in the figure, during the communication process of the industrial control system, a packet capture tool is used to capture network data packets to generate a dataset, which then forms a seed file. These seed files are preprocessed by the preprocessing module in the vulnerability mining tool and then used as the training dataset for the Mix-GAN model. After model training, the model parameters are saved, and the generation module can generate data that conforms to the trained model parameters. This data is then mutated using a mutation strategy to generate test cases. These test cases are used to fuzz test the system under test, and the fuzz testing results are evaluated by analyzing the system's feedback information.

[0037] The present invention proposes a Modbus TCP protocol test case generation method based on a generative adversarial network. The flow chart is as follows: Figure 2 As shown, the following steps are included:

[0038] S1: Extract target protocol message data from the captured network protocol messages and pre-process the data as a model training dataset;

[0039] Preferably, in step S1: extracting target protocol message data from the captured network protocol message and preprocessing the data as a model training data set, specifically includes the following sub-steps:

[0040] S101: Capture protocol data packets when the host computer and PLC are communicating normally;

[0041] S102: extracting Modbus TCP protocol data from the data packet;

[0042] S103: Convert the Modbus TCP protocol data into binary format;

[0043] S104: Align the data by adding '0' after the short data to align all the data;

[0044] S105: After all data are aligned, one-hot encoding is used to convert the data format to form a training data set;

[0045] S2: If Figure 3 As shown in the figure, by combining the generative adversarial network, the bidirectional long short-term memory network and the self-attention mechanism, an efficient training model is constructed and the training data set is used to train the model;

[0046] Preferably, in step S2: by combining the generative adversarial network, the bidirectional long short-term memory network and the self-attention mechanism, an efficient training model is constructed, and the model is trained using the training data set and test cases are generated, which specifically includes the following sub-steps:

[0047] S201: Considering that Bi-LSTM can effectively capture long-term dependencies and contextual information in sequence data, thus generating more accurate and diverse sequences, and the self-attention mechanism helps the model better handle long sequences and global dependencies, improving the quality and diversity of generated sequences. Therefore, the Bi-LSTM layer, the self-attention mechanism layer, and the fully connected layer are used as the generator of the generative adversarial network, and the Bi-LSTM layer, the fully connected layer, and the Softmax layer are used as the discriminator of the generative adversarial network. Set some model parameters, such as the number of data selected at a time. , learning rate , pre-training times , number of adversarial training , minimum error .in =100, =0.0001, =100, =500, =1e9 can be adjusted based on the test results of different PLCs. Adjust the value of ;

[0048] S202: The training dataset is used as input for pre-training of the generator. After the generator training is completed, fake data is generated. The fake data and the training dataset are used as the dataset of the discriminator in a 1:1 ratio for pre-training of the discriminator.

[0049] S203: Perform adversarial training. First, fix the discriminator and train the generator. The generator generates false data samples that are similar to real data, and inputs the false data samples generated by the generator into the discriminator together with the real data. The discriminator will calculate a probability value through the loss function, indicating the possibility that the input sample is real data. According to the loss function of the generator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the generator are updated to improve its ability to generate sample data that is highly similar to the real data. Finally, fix the generator and train the discriminator. The discriminator receives real data samples and false data samples generated by the generator at the same time. According to the loss function of the discriminator, it calculates the probability value that the false data sample can be regarded as real data. The goal is to minimize this loss function. According to the loss function of the discriminator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the discriminator are updated to improve its discrimination ability. These two steps are repeated until the stopping condition is reached. The discriminator loss function is as follows:

[0050] ;

[0051] in is the probability value, n is the total number of data selected at a time, is the discriminator’s prediction for the i-th sample, is the i-th true label.

[0052] S3: Determine whether the training model end conditions are met. If so, save the parameters of the generator model and generate test cases based on the parameters. Otherwise, jump to S2. The termination conditions are as follows:

[0053] ;

[0054] in is the probability calculated by the discriminator loss function, Minimum error

[0055] S4: Use the mutation strategy in the genetic algorithm to mutate the test case;

[0056] Preferably, in step S4: adopting the mutation strategy in the genetic algorithm to mutate the test case, specifically comprising the following sub-steps:

[0057] S401: Binary encode the initial test case;

[0058] S402: Perform different mutation operations based on the abnormal code of the test case. For example, a normal individual performs a right shift operation or an XOR operation. Abnormal code 1 will change the function code to the initial function code and then perform an XOR operation. Others will perform an XOR operation.

[0059] S403: Decoding the test case after mutation;

[0060] S5: Send the test case to the PLC under test through the socket to monitor the status of the tested system;

[0061] Preferably, in step S5, the data packet returned by the tested system is used to observe the state of the tested system, and a response code is obtained using a parsing module.

[0062] This paper introduces a Bi-LSTM and self-attention mechanism-optimized generator and a Bi-LSTM-optimized discriminator as training models. These models can more accurately capture long-term dependencies, thereby addressing the issue of incomplete protocol format and feature learning. After training, a generation module generates test cases that conform to the protocol format, increasing the test case response rate. Furthermore, a genetic algorithm mutation strategy is designed to increase test case diversity, avoid test case redundancy, and improve the efficiency of fuzz testing.

[0063] The above embodiments are merely preferred technical solutions of the present invention and should not be construed as limiting the present invention. The scope of protection of the present invention shall be the technical solutions set forth in the claims, including equivalent alternatives to the technical features of the technical solutions set forth in the claims. In other words, equivalent alternatives and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A Modbus TCP protocol test case generation method based on generative adversarial networks, characterized in that: The following steps are involved: S1: Extract target protocol message data from the captured network protocol messages and pre-process the data as a model training dataset; S2: By combining generative adversarial networks, bidirectional long short-term memory networks, and self-attention mechanisms, a training model was constructed and trained using the training dataset. Model training involves using the Bi-LSTM layer, self-attention mechanism layer, and fully connected layer as the generator of the generative adversarial network, and the Bi-LSTM layer, fully connected layer, and Softmax layer as the discriminator of the generative adversarial network; S3: Determine whether the training model end conditions are met. If so, save the parameters of the generator model and generate test cases based on the parameters. Otherwise, jump to S2. S4: Use the mutation strategy in the genetic algorithm to mutate the test case; S5: Send the test case to the PLC under test through the socket, monitor the status of the tested system and obtain the response code.

2. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that, The step S1 comprises the following steps: S101: Capture protocol data packets when the host computer and PLC are communicating normally; S102: extracting Modbus TCP protocol data from the data packet; S103: Convert the Modbus TCP protocol data into binary format; S104: Align the data by adding '0' after the short data to align all the data; S105: After all the data are aligned, one-hot encoding is used to convert the format to form a training data set.

3. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that, The step S2 comprises the following steps: S201: Set model parameters and the number of data selected at a time , learning rate , pre-training times , number of adversarial training , minimum error ; S202: The training dataset is used as input for pre-training of the generator. After the generator training is completed, fake data is generated; the fake data and the training dataset are used as the dataset of the discriminator in a 1:1 ratio for pre-training of the discriminator. S203: Conduct adversarial training. First, fix the discriminator and train the generator. The generator generates false data samples similar to the real data. The false data samples generated by the generator are input into the discriminator together with the real data. The discriminator calculates a probability value through the loss function. , indicating the possibility that the input sample is real data; according to the loss function of the generator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the generator are updated to improve its ability to generate sample data that is highly similar to the real data; finally, the generator is fixed and the discriminator is trained; the discriminator receives real data samples and false data samples generated by the generator at the same time, and calculates the probability value of the false data sample as real data according to the loss function of the discriminator. The goal is to minimize this loss function. According to the loss function of the discriminator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the discriminator are updated to improve its discrimination ability; these two steps are repeated until the stopping condition is reached. The discriminator loss function is as follows: ； in is the probability value, n is the total number of data selected at a time, is the discriminator’s prediction for the i-th sample, is the i-th true label.

4. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that, In step S3, when the probability value calculated by the discriminator loss function is When the error is less than the minimum error Min-loss, the training ends and the generator model parameters are saved. Test cases are generated according to the parameters. Otherwise, the training continues. The termination conditions are as follows: ； in is the probability calculated by the discriminator loss function, is the minimum error.

5. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that, The step S4 comprises the following steps: S401: Binary encode the initial test case; S402: Perform different mutation operations based on the abnormal code of the test case, including right shift operation or XOR operation for normal individuals. Abnormal code 1 will change the function code to the initial function code and then perform XOR operation. Others will perform XOR operation. S403: Decode the test case after mutation.

6. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that, In step S5, the data packet returned by the tested system is used to observe the state of the tested system, and a response code is obtained using a parsing module.

Images