Modbus TCP protocol test case generation method based on generative adversarial network

Abstract

A Modbus TCP protocol test case generation method based on a generative adversarial network comprises the following steps: S1, extracting target protocol message data from a captured network protocol message, and preprocessing the data as a model training data set; s2, constructing an efficient training model by combining a generative adversarial network, a bidirectional long-short-term memory network and a self-attention mechanism, and training the model by using a training data set; s3, judging whether a training model ending condition is met or not, if so, storing parameters of the generator model, and generating a test case according to the parameters, otherwise, skipping to S2; s4, performing variation on the test case by adopting a variation strategy in a genetic algorithm; and S5, the test case is sent to the tested PLC through the Socket, the state of the tested system is monitored, and a response code is obtained. The problems of incomplete format learning, low response rate and high redundancy in Modbus TCP protocol test case generation are solved, and the fuzzy test efficiency is improved.

Description

Technical Field

[0001] The invention relates to the field of fuzzy technology, and in particular to a method for generating Modbus TCP protocol test cases based on a generative adversarial network. Background Art

[0002] With the continuous evolution of the industrial Internet and the acceleration of the deep integration of informatization and industrialization, the boundary between traditional industrial control systems and the Internet is becoming increasingly blurred. Industrial control systems that were originally designed only for intranet communication are gradually connected to the Internet. While this access greatly improves production efficiency and openness, it also exposes traditional industrial control systems to increasingly severe Internet security threats. The core of fuzz testing is to identify and locate potential vulnerabilities or defects in software or equipment by injecting unexpected and abnormal input data into the target system and closely monitoring the response of the system. Protocol fuzz testing has become a highly targeted and effective testing method. This method simulates malicious or abnormal network communication behaviors through carefully designed malformed data packets, sends these data packets to the industrial control equipment under test, and closely monitors the operating status and response of the equipment to detect and expose possible security vulnerabilities in the protocol processing of industrial control equipment.

[0003] The existing solution is: in protocol fuzz testing, the sequence generative adversarial network model is introduced to optimize the generation of test cases and improve the comprehensiveness and efficiency of the test. The SEQ-GAN model combines the ideas of generative adversarial networks and reinforcement learning, and is particularly suitable for processing discrete sequence data generation tasks. The core of this model is to use the adversarial training mechanism of GAN to generate realistic sequence data, while borrowing the policy gradient method of reinforcement learning to solve the problem that discrete data is difficult to update through gradients. However, the current fuzz testing also has its own problems. However, the current fuzz testing has its limitations, mainly manifested in the incomplete learning of protocol formats and features, low test case response rate and high redundancy. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a Modbus TCP protocol test case generation method based on a generative adversarial network, which solves the problems of incomplete format learning, low response rate and high redundancy in Modbus TCP protocol test case generation and improves the efficiency of fuzzy testing.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: a Modbus TCP protocol test case generation method based on a generative adversarial network, comprising the following steps: S1: extract target protocol message data from the captured network protocol message, and pre-process the data as a model training data set; S2: By combining generative adversarial networks, bidirectional long short-term memory networks, and self-attention mechanisms, an efficient training model is constructed and trained using the training dataset; S3: Determine whether the training model end condition is met. If so, save the parameters of the generator model and generate test cases based on the parameters. Otherwise, jump to S2. S4: Use the mutation strategy in the genetic algorithm to mutate the test case; S5: Send the test case to the PLC under test through Socket, monitor the status of the tested system and obtain the response code.

[0006] Preferably, step S1 comprises the following steps: S101: Capture protocol data packets when the host computer and PLC communicate normally; S102: extracting Modbus TCP protocol data from the data packet; S103: Convert Modbus TCP protocol data into binary; S104: align the data by adding '0' after the short data to align all the data; S105: After all the data are aligned, One-Hot encoding is used to convert the format to form a training data set.

[0007] Preferably, step S2 comprises the following steps: S201: Considering that Bi-LSTM can effectively capture long-term dependencies and contextual information in sequence data, thereby generating more accurate and diverse sequences, and the self-attention mechanism helps the model better handle long sequences and global dependencies, improving the quality and diversity of generated sequences; therefore, the Bi-LSTM layer, the self-attention mechanism layer, and the fully connected layer are used as the generator of the generative adversarial network, and the Bi-LSTM layer, the fully connected layer, and the Softmax layer are used as the discriminator of the generative adversarial network; set the model parameters, the number of data selected at a time , learning rate , pre-training times , adversarial training times , minimum error . These model parameters are pre-set values; S202: The training data set is used as the input of the generator for pre-training. After the generator training is completed, false data is generated; the false data and the training data set are used as the data set of the discriminator in a ratio of 1:1 for pre-training of the discriminator; S203: Perform adversarial training. First, fix the discriminator and train the generator. The generator generates false data samples similar to the real data, and inputs the false data samples generated by the generator and the real data into the discriminator. The discriminator calculates a probability value through the loss function. , indicating the possibility that the input sample is real data; according to the loss function of the generator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the generator are updated to improve the sample data generated by it that is highly similar to the real data; finally, the generator is fixed and the discriminator is trained; the discriminator receives the real data samples and the false data samples generated by the generator at the same time, and calculates the probability value that the false data samples can be regarded as real data according to the loss function of the discriminator. The goal is to minimize this loss function. According to the loss function of the discriminator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the discriminator are updated to improve its discrimination ability; these two steps are repeated until the stopping condition is reached. The discriminator loss function is as follows: ; in is the probability value, n is the total number of data selected at one time, is the discriminator’s prediction for the i-th sample, is the i-th true label.

[0008] Preferably, in step S3, when the probability value calculated by the discriminator loss function is When it is less than the minimum error Min-loss, the training ends and the generator model parameters are saved, and test cases are generated according to the parameters, otherwise the training continues. The termination conditions are as follows: ; in is the probability calculated by the discriminator loss function, is the minimum error.

[0009] Preferably, step S4 comprises the following steps: S401: Binary encode the initial test case; S402: Perform different mutation operations according to the abnormal code of the test case, such as right shift operation or XOR operation for normal individuals, abnormal code 1 will change the function code to the initial function code, and then perform XOR operation, and all others will perform XOR operation; S403: Decode the test case after mutation.

[0010] Preferably, in step S5, the response code is obtained by using the parsing module through the data packet returned by the tested system and observing the status of the tested system.

[0011] The present invention provides a Modbus TCP protocol test case generation method based on a generative adversarial network, and uses a generative adversarial network to generate a Modbus TCP protocol test case generation method, thereby improving the effectiveness and efficiency of the test. By integrating Bi-LSTM and self-attention mechanism into a generative adversarial network model, the present invention successfully solves the problem of incomplete protocol format and feature learning in traditional fuzzy testing, so that the generated test case can more accurately simulate the actual communication behavior of the Modbus TCP protocol, thereby effectively exposing potential security vulnerabilities. In addition, the present invention introduces a mutation strategy of a genetic algorithm, performs intelligent mutation on test cases, greatly reduces the redundancy of test cases, and ensures the diversity and pertinence of test data. This optimization not only reduces the complexity of the test process, but also significantly improves the response rate of test cases. The Modbus TCP protocol test case constructed by the present invention has a high degree of protocol compliance, low redundancy and high response rate, which greatly improves the efficiency of fuzzy testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention will be further described below in conjunction with the accompanying drawings and embodiments: Figure 1 It is a test case generation system architecture diagram of the Modbus TCP protocol provided by the present invention; Figure 2 It is a flow chart of a test case generation method for the Modbus TCP protocol provided by the present invention; Figure 3 It is a structural diagram of the training model provided by the present invention. DETAILED DESCRIPTION

[0013] The present invention designs a Modbus TCP protocol test case generation method architecture based on generative adversarial network, which is implemented in Python. The architecture diagram is as follows: Figure 1 As shown in the figure. During the communication process of the industrial control system, a packet capture tool is used to capture network data packets to generate a data set and form a seed file. These seed files are preprocessed by the preprocessing module in the vulnerability mining tool and used as the training data set of the Mix-GAN model. After the model is trained, the model parameters are saved, and the generation module can generate data that meets the trained model parameters. Then, these data are mutated in combination with the mutation strategy to form test cases. These test cases are used to perform fuzz testing on the tested system, and the results of the fuzz testing are evaluated by analyzing the feedback information of the system.

[0014] The present invention proposes a Modbus TCP protocol test case generation method based on a generative adversarial network, the flow chart is as follows: Figure 2 As shown, the following steps are included: S1: extract target protocol message data from the captured network protocol message, and pre-process the data as a model training data set; Preferably, in step S1: extracting target protocol message data from captured network protocol messages, and preprocessing the data as a model training data set, specifically comprises the following sub-steps: S101: Capture protocol data packets when the host computer and PLC communicate normally; S102: extracting Modbus TCP protocol data from the data packet; S103: Convert Modbus TCP protocol data into binary; S104: align the data by adding '0' after the short data to align all the data; S105: After all the data are aligned, One-Hot encoding is used to convert the format to form a training data set; S2: Figure 3 As shown in the figure, by combining the generative adversarial network, the bidirectional long short-term memory network and the self-attention mechanism, an efficient training model is constructed, and the model is trained using the training data set; Preferably, in step S2: by combining the generative adversarial network, the bidirectional long short-term memory network and the self-attention mechanism, an efficient training model is constructed, and the model is trained using the training data set and test cases are generated, which specifically includes the following sub-steps: S201: Considering that Bi-LSTM can effectively capture long-term dependencies and contextual information in sequence data, thereby generating more accurate and diverse sequences, and the self-attention mechanism helps the model better handle long sequences and global dependencies, improving the quality and diversity of generated sequences. Therefore, the Bi-LSTM layer, the self-attention mechanism layer, and the fully connected layer are used as the generator of the generative adversarial network, and the Bi-LSTM layer, the fully connected layer, and the Softmax layer are used as the discriminator of the generative adversarial network. Set some model parameters, such as the number of data selected at a time. , learning rate , pre-training times , adversarial training times , minimum error .in =100, =0.0001, =100, =500, =1e9 can be adjusted according to the test results of different PLCs. The value of is adjusted. ; S202: The training data set is used as the input of the generator for pre-training. After the generator training is completed, fake data is generated. The fake data and the training data set are used as the data set of the discriminator in a 1:1 ratio for pre-training of the discriminator; S203: Perform adversarial training. First, fix the discriminator and train the generator. The generator generates false data samples that are similar to real data, and inputs the false data samples generated by the generator into the discriminator together with the real data. The discriminator will calculate a probability value through the loss function, indicating the possibility that the input sample is real data. According to the loss function of the generator, the gradient of the parameters is calculated through the back propagation algorithm, and the parameters of the generator are updated to improve its ability to generate sample data that is highly similar to the real data. Finally, fix the generator and train the discriminator. The discriminator receives real data samples and false data samples generated by the generator at the same time. According to the loss function of the discriminator, it calculates the probability value that the false data sample can be regarded as real data. The goal is to minimize this loss function. According to the loss function of the discriminator, the gradient of the parameters is calculated through the back propagation algorithm, and the parameters of the discriminator are updated to improve its discrimination ability. These two steps are repeated until the stopping condition is reached. The discriminator loss function is as follows: ; in is the probability value, n is the total number of data selected at one time, is the discriminator’s prediction for the i-th sample, is the i-th true label.

[0015] S3: Determine whether the training model end condition is met. If so, save the parameters of the generator model and generate test cases based on the parameters. Otherwise, jump to S2. The termination conditions are as follows: ; in is the probability calculated by the discriminator loss function, Minimum error S4: Use the mutation strategy in the genetic algorithm to mutate the test case; Preferably, in step S4: adopting a mutation strategy in a genetic algorithm to mutate the test case, specifically comprising the following sub-steps: S401: Binary encode the initial test case; S402: Perform different mutation operations according to the abnormal code of the test case, such as right shift operation or XOR operation for normal individuals, abnormal code 1 will change the function code to the initial function code, and then perform XOR operation, and all others will perform XOR operation; S403: Decoding the test case after mutation; S5: Send the test case to the PLC under test through Socket to monitor the status of the system under test; Preferably, in step S5, the response code is obtained by using the parsing module through the data packet returned by the tested system and observing the status of the tested system.

[0016] The present invention introduces Bi-LSTM and self-attention mechanism optimization generator and introduces Bi-LSTM optimization discriminator as training models, which can more accurately capture long-term dependencies, thereby solving the problem of incomplete protocol format and feature learning. After the training is completed, the generation module generates test cases that conform to the protocol format to increase the response rate of the test cases. In addition, the mutation strategy of the genetic algorithm is designed to increase the diversity of test cases, avoid test case redundancy, and improve the efficiency of fuzzy testing.

[0017] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A Modbus TCP protocol test case generation method based on a generative adversarial network, characterized in that: The following steps are involved: S1: extract target protocol message data from the captured network protocol message, and pre-process the data as a model training data set; S2: By combining generative adversarial networks, bidirectional long short-term memory networks, and self-attention mechanisms, an efficient training model is constructed and trained using the training dataset; S3: Determine whether the training model end condition is met. If so, save the parameters of the generator model and generate test cases based on the parameters. Otherwise, jump to S2. S4: Use the mutation strategy in the genetic algorithm to mutate the test case; S5: Send the test case to the PLC under test through Socket, monitor the status of the tested system and obtain the response code.

2. According to a Modbus TCP protocol test case generation method based on a generative adversarial network as described in claim 1, it is characterized in that, The step S1 comprises the following steps: S101: Capture protocol data packets when the host computer and PLC communicate normally; S102: extracting Modbus TCP protocol data from the data packet; S103: Convert Modbus TCP protocol data into binary; S104: align the data by adding '0' after the short data to align all the data; S105: After all the data are aligned, One-Hot encoding is used to convert the format to form a training data set.

3. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that: The step S2 comprises the following steps: S201: Considering that Bi-LSTM can effectively capture long-term dependencies and contextual information in sequence data, thereby generating more accurate and diverse sequences, and the self-attention mechanism helps the model better handle long sequences and global dependencies, improving the quality and diversity of generated sequences; therefore, the Bi-LSTM layer, the self-attention mechanism layer, and the fully connected layer are used as the generator of the generative adversarial network, and the Bi-LSTM layer, the fully connected layer, and the Softmax layer are used as the discriminator of the generative adversarial network; set the model parameters, the number of data selected at a time , learning rate , pre-training times , adversarial training times , minimum error ; S202: The training data set is used as the input of the generator for pre-training. After the generator training is completed, false data is generated; the false data and the training data set are used as the data set of the discriminator in a ratio of 1:1 for pre-training of the discriminator; S203: Perform adversarial training. First, fix the discriminator and train the generator. The generator generates false data samples similar to the real data, and inputs the false data samples generated by the generator and the real data into the discriminator. The discriminator calculates a probability value through the loss function. , indicating the possibility that the input sample is real data; according to the loss function of the generator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the generator are updated to improve the sample data generated by it that is highly similar to the real data; finally, the generator is fixed and the discriminator is trained; the discriminator receives the real data samples and the false data samples generated by the generator at the same time, and calculates the probability value that the false data samples can be regarded as real data according to the loss function of the discriminator. The goal is to minimize this loss function. According to the loss function of the discriminator, the gradient of the parameters is calculated by the back propagation algorithm, and the parameters of the discriminator are updated to improve its discrimination ability; these two steps are repeated until the stopping condition is reached. The discriminator loss function is as follows: ； in is the probability value, n is the total number of data selected at one time, is the discriminator’s prediction for the i-th sample, is the i-th true label.

4. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that: In step S3, when the probability value calculated by the discriminator loss function When the error is less than the minimum error Min-loss, the training ends and the generator model parameters are saved. Test cases are generated according to the parameters. Otherwise, the training continues. The termination conditions are as follows: ； in is the probability calculated by the discriminator loss function, is the minimum error.

5. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that: The step S4 comprises the following steps: S401: Binary encode the initial test case; S402: Perform different mutation operations according to the abnormal code of the test case, such as right shift operation or XOR operation for normal individuals, abnormal code 1 will change the function code to the initial function code, and then perform XOR operation, and all others will perform XOR operation; S403: Decode the test case after mutation.

6. A Modbus TCP protocol test case generation method based on a generative adversarial network according to claim 1, characterized in that: In step S5, the data packet returned by the tested system is used to observe the state of the tested system, and a response code is obtained by using a parsing module.

Images