Industrial control system threat trapping method based on machine learning algorithm

Abstract

This paper discloses a threat trapping method for industrial control systems based on machine learning algorithms. To address the complex types and high-dimensionality of abnormal traffic in power industrial control systems, a random forest algorithm is introduced to design a data detection module for a dynamic honeynet system, improving the accuracy of abnormal traffic identification. Furthermore, the deep reinforcement learning (DQN) algorithm is used to help the honeynet system dynamically adjust its strategy in response to different attack behaviors. Experimental results demonstrate that this method can effectively enhance the network security protection capabilities of industrial control systems.

Description

Technical Field

[0001] The present invention relates to the technical field of risk assessment, and in particular to a method for luring threats to an industrial control system based on a machine learning algorithm. Background Art

[0002] As critical infrastructure in modern society, the security of industrial control systems is paramount. However, with the evolution of Industry 4.0, the unprecedented integration of next-generation information technology and industrialization is directly driving the transformation of traditional closed industrial control systems toward openness and intelligence, making them increasingly vulnerable to cyberattacks.

[0003] With the rapid development of global network informatization, cybersecurity threats are constantly evolving, and attack methods are becoming increasingly sophisticated. Traditional network security measures are facing a severe test of their effectiveness in the face of rapidly evolving attack methods, especially advanced persistent threats that exploit system vulnerabilities and unknown attack patterns. To overcome the passive position of defenders in cyberspace confrontations, security personnel are turning their attention to honeynet security defense technology based on active defense. This technology deploys honeypots and honeynets that mimic the characteristics of industrial control systems to lure attackers, capture and analyze attack behaviors, and update defense strategies to strengthen the security protection capabilities of industrial control systems.

[0004] Regarding honeynet technology for industrial control system security, Wang Yuzhen et al. designed and implemented a reinforcement learning-based smart grid honeypot framework, SGPot. This framework accurately simulates the constant system parameters of real power industry equipment, achieving a high-fidelity representation of the smart substation control terminal. To address the challenges of current defense methods against network attacks, which often suffer from high probability of successful intrusion and short attack times, Wu Nanxu et al. proposed an active defense solution combining the K-means algorithm and honeypot technology. Lin Zhi et al. designed a honeypot framework based on industrial control equipment simulation to target high-risk functions of the Modbus protocol and its subsidiary UMAS protocol in industrial control systems. This framework accurately simulates UMAS protocol messages and enhances the honeypot's entrapment capabilities. G. Wagener defined the configuration and behavior of high-interaction honeypots based on game theory concepts. As a variant of reinforcement learning, this approach aims to help honeypots adopt optimal defensive behaviors in the face of network attacks. Pauna et al. first developed an adaptive medium-interaction honeypot system, called RASSH, which simulates an SSH server and uses machine learning algorithms to intelligently interact with attackers. Yamamoto designed and implemented a framework for automatically generating IoT honeypots using FirmPot firmware. Seamus Dowling proposed a machine learning-based adaptive honeypot designed to intelligently respond to attack commands through learning. Touch et al. innovatively proposed the reinforcement learning honeypot architecture Asgard and its derivative Midgard. Lopezy et al. significantly improved attacker interaction within an adaptive honeypot environment by combining the word2vec model with deep reinforcement learning techniques.

[0005] However, the static nature of traditional honeynet technology limits its effectiveness in complex network environments. Most honeypots employ static defenses, requiring manual deployment and configuration. This leads to resource optimization and efficiency issues. Therefore, a more efficient threat trapping model is urgently needed to reduce resource waste and improve system performance. Summary of the Invention

[0006] In view of the shortcomings of the above-mentioned prior art, the present invention provides an industrial control system threat trapping method based on a machine learning algorithm, which comprises the following steps:

[0007] Step 1: Modify the initial configuration file of the Conpot honeypot based on the characteristics of each device and protocol in the industrial control system;

[0008] Step 2: Deploy multiple honeypot nodes of different types in the industrial control network to simulate different types of industrial control equipment;

[0009] Step 3: Build a decoy model on each honeypot node;

[0010] Step 4: Use the trapping model to judge the data traffic entering the honeypot node, and dynamically execute the configured strategy when capturing abnormal data.

[0011] Furthermore, the specific steps of step 3 to establish the trapping model include:

[0012] Step 3.1: Based on the CIC-IDS-2017 dataset, construct a traffic classification sample set;

[0013] Step 3.2: Build a random forest model and train and test the model using the traffic classification sample set;

[0014] Step 3.3: Use the output of the random forest model as the input of the DQN model;

[0015] Step 3.4: The DQN model detects the input traffic data label, obtains different rewards based on the label type, calculates the target Q-value function based on the reward, optimizes the DQN model parameters by minimizing the loss function, and constructs a threat trapping model.

[0016] Furthermore, the specific steps of step 3.1 include:

[0017] Step 3.1.1: Delete the dirty data with NaN and Infinite feature values ​​in the two columns of stream byte rate and stream packet rate in the original dataset;

[0018] Step 3.1.2: Randomly select one-third of the benign traffic data as the normal traffic data sample;

[0019] Step 3.1.3: Filter out non-benign traffic from the remaining data in the original dataset as abnormal traffic samples;

[0020] Step 3.1.4: Merge the normal traffic data samples and abnormal traffic samples and remove all dirty data rows to obtain the traffic classification sample set.

[0021] Furthermore, the specific steps of step 3.2 include:

[0022] Step 3.2.1: Standardize the data in the traffic classification sample set according to the following formula:

[0023]

[0024] Among them, x i represents the original data, z i represents the data after standardization, μ represents the mean of the original data, and σ represents its standard deviation;

[0025] Step 3.2.2: Divide the normalized traffic classification sample set into a training set and a test set in a ratio of 7:3;

[0026] Step 3.2.3: Encode the labels of the sample data in the training set and test set;

[0027] Step 3.2.4: Train the random forest model on the training set, update the model parameters, and optimize the model;

[0028] Step 3.2.5: Test the trained random forest model on the test set;

[0029] Step 3.2.6: Build a random forest model.

[0030] Furthermore, the specific steps of step 3.4 include:

[0031] Step 3.4.1: Obtain traffic data after random forest model detection;

[0032] Step 3.4.2: The DQN model determines the label in the traffic data. If the label is "BENIGN", the current traffic data is judged to be abnormal traffic and proceeds to step 3.4.3; otherwise, the current traffic data is normal traffic and a corresponding negative reward is obtained;

[0033] Step 3.4.3: Execute the abnormal traffic action. If the execution is successful, you will get the corresponding positive reward; otherwise, you will get the corresponding negative reward;

[0034] Step 3.4.4: Calculate the target Q value based on the reward value, minimize the loss function, and determine whether the loss value of the loss function is less than or equal to the preset threshold. If not, use the gradient descent method to update the DQN parameters; if so, output the constructed DQN model.

[0035] Furthermore, the step of calculating the target Q value in step 3.4.4 is as follows: based on the state s′ corresponding to the abnormal traffic data, the Q value corresponding to the action a′ of modifying the honeypot fingerprint information is calculated by the following formula:

[0036]

[0037] Where γ represents the discount factor, which is used to balance the weight between current rewards and future potential rewards; θ － represents the parameters of the target network; r i It represents the immediate reward obtained by the agent after executing action a in state s.

[0038] Furthermore, the loss function is:

[0039]

[0040] Among them, s represents a specific state, a represents the execution action, θ represents the parameters of the DQN neural network, and N represents the number of samples.

[0041] Therefore, the present invention adopts the above-mentioned industrial control system threat trapping method based on machine learning algorithm, which has the following beneficial effects:

[0042] This paper proposes a machine learning-based threat trapping technology for industrial control systems. An industrial control system simulation environment is built on Conpot. Real-time traffic detection and classification are performed based on the random forest model. The CIC-IDS-2017 dataset is used to train the random forest algorithm model to improve the accuracy of abnormal traffic identification. By introducing the DQN algorithm, dynamic configuration of honeynet strategies is achieved, effectively solving the problem of the lack of dynamicity in traditional honeynets.

[0043] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is the principle diagram of the random forest algorithm.

[0045] Figure 2 This is the basic structure diagram of DQN.

[0046] Figure 3 This is the basic processing flow chart of random forest.

[0047] Figure 4 This is the basic flow chart of DQN.

[0048] Figure 5 It is the prediction result diagram in the embodiment.

[0049] Figure 6 4 is a confusion matrix diagram of the prediction results in the embodiment.

[0050] Figure 7 The following is a graph showing the accuracy of sample sets with different proportions in the embodiment;

[0051] Figure 8 (a)-(b) are the relationship diagrams of rewards and ε with training iterations during the training iteration process;

[0052] Figure 9 This is the Nmap fingerprint scan image.

[0053] Figure 10 This is the confusion matrix of the optimized model for detecting “Infiltration” and “Heartbleed” attacks in the embodiment. DETAILED DESCRIPTION

[0054] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art will make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of the application.

[0055] The present invention proposes a threat trapping method for industrial control systems based on machine learning. By combining typical machine learning algorithms, a threat trapping model for industrial control systems consisting of multiple honeypots is constructed. This threat trapping model solves the problem of lack of dynamics in traditional honeynets.

[0056] Next, we will introduce the basic theory of constructing this model.

[0057] 1. Random Forest Algorithm

[0058] Random forest is a supervised ensemble model based on multiple independent decision trees. During the training process, k samples are extracted from the original training set N with replacement using the Bootstrap method to form k independent training sets, and then k decision trees are constructed. For each decision tree, a small number of features are randomly selected from all features for optimal segmentation. In the process of judging the input data, each decision tree will make a judgment and form a classification result. According to the principle of minority obeys majority, the one with the most classification results is the final output judgment result. Its basic principle is as follows: Figure 1 shown.

[0059] 2. DQN algorithm

[0060] The DQN (Deep Q-Network) algorithm is a reinforcement learning algorithm based on deep learning. Its core idea is to integrate deep learning and Q-learning. DQN is not only suitable for high-dimensional, continuous state spaces, but also can efficiently learn and make decisions in unknown environments, providing effective solutions to complex problems.

[0061] DQN's intelligent system uses deep neural networks to learn the state-value function Q(s,a) in a complex environment, where s represents the state of the environment observed by the agent and a represents the action the agent takes within that state. The Q-value function is an evaluation metric used to measure the expected cumulative reward that the agent can obtain by taking action a in a specific state s under a specific strategy π. DQN uses neural networks to approximate this function, as shown in the following formula:

[0062] Q(s, a; θ)≈Q * (s,a) (1)

[0063] Where Q*(s,a) represents the Q value of the optimal strategy, Q(s,a;θ) represents the approximate Q value output by the neural network, and θ represents the parameters of the neural network.

[0064] By using the target network to calculate the target value, the stability of learning is increased, and the target value is defined as follows:

[0065]

[0066] Among them, s′ represents the next state to be entered at the next moment; a′ represents the action at the next moment; γ represents the discount factor, which is used to balance the weight between the current reward and the potential future reward; θ － represents the parameters of the target network; r i It represents the immediate reward obtained by the agent after executing action a in state s; the target value represents y i The maximum expected reward that can be obtained by taking the optimal action in state s′.

[0067] DQN will also establish a loss function, which is the mean square error (MSE) between the target value and the predicted Q value. This loss function is used to evaluate the deviation between the predicted Q value and the target Q value. The core goal of machine learning training is to minimize this deviation. The loss function is:

[0068]

[0069] Where N represents the number of samples;

[0070] Finally, in order to minimize the loss function, the gradient descent method is used to update the parameters θ of the neural network. The parameter update rule is as follows:

[0071]

[0072] Where α is the learning rate, is the gradient of the loss function with respect to the parameters θ.

[0073] In DQN, the environment is viewed as a dynamic and complex system, and the agent needs to continuously interact with it to collect experience data. This experience data includes the state observed by the agent, the actions performed, the immediate rewards obtained, and the next state. DNN uses this data to learn and approximate the Q-value function and continuously optimize the network parameters by minimizing the prediction error. The entire structure of DQN is as follows Figure 2 shown.

[0074] 3. Build an industrial control dynamic honeynet system based on machine learning

[0075] (1) Conpot honeypot

[0076] Conpot is an open-source, low-interaction industrial control honeypot developed and designed by the Mushroom Security team. It can create a simulated industrial control system environment by emulating industrial control devices such as programmable logic controllers and SCADA servers, and industrial control protocols such as S7comm, Modbus, and SNMP. The Conpot honeypot utilizes a flexible and modular design, allowing users to customize its configuration based on the general configuration. By adding customized templates or protocols, users can simulate different scenarios. Table 1 shows the core modules of the Conpot honeypot and their functions.

[0077] Table 1 Conpot modules and functions

[0078]

[0079] In Conpot's default template, the template.xml file emulates a Siemens S7-200 series PLC. This file not only mimics the device's copyright information but also forges its CPU information. However, due to a flaw in the file, which contains Conpot's unique fingerprint "88111222," it can be easily identified by attackers and labeled as a "honeypot."

[0080] Therefore, based on the characteristics of the devices and protocols in industrial control systems, the initial configuration file of the Conpot honeypot was modified. This included adding or removing custom keys, or customizing settings based on requirements. Taking the Siemens S7-300 PLC device in a microgrid as an example, Conpot was simulated and configured. Table 2 details the modified key parameters.

[0081] Table 2 Siemens S7-300 simulation parameter information

[0082]

[0083] A single honeypot can only simulate a specific device or service within an industrial control system, making it ineffective for attracting attackers. Therefore, to simulate a more realistic and attractive industrial control network environment, it's necessary to deploy multiple honeypot nodes of different types simultaneously, each simulating different types of industrial control equipment and enabling communication services between honeypot nodes. This way, when an attacker detects a honeypot node, they'll also detect data flows with other honeypot nodes, making it impossible to determine whether it's a trap or not, effectively enhancing the system's deception capabilities.

[0084] (2) CIC-IDS-2017 dataset

[0085] The CIC-IDS-2017 dataset covers 14 common attack types, including web attacks, penetration tests, brute force attacks, DDoS attacks, and port scans. The dataset consists of eight files stored in CSV format, as shown in Table 3. Each file contains complete network traffic data, including 78 feature columns and one label column.

[0086] Table 3 CIC-IDS-2017 dataset files

[0087]

[0088] ① Data preprocessing

[0089] To construct the traffic classification sample set, we first cleaned the "Flow Bytes / s" and "Flow Packets / s" columns, containing some "NaN" and "Infinite" feature values. We then randomly selected one-third of the data from "Monday," which consisted entirely of benign traffic, as normal traffic samples. We also selected non-benign traffic from the remaining files as abnormal traffic samples. Finally, we merged these two types of samples, removing all the dirty data rows, to form a complete sample set for analysis and model training. The number of samples with different labels in this traffic classification sample set is shown in Table 4.

[0090] Table 4 Number of samples for each label

[0091]

[0092] ② Standardization

[0093] Since the traffic classification sample set is large, standardizing the data to make it normal distribution (mean 0, standard deviation 1) can effectively reduce the training time of the machine learning model. The formula for standardizing the data is:

[0094]

[0095] Among them, x i represents the original data, z i represents the data after standardization, μ represents the mean of the original data, and σ represents its standard deviation.

[0096] ③ Sample set division

[0097] After completing the above two steps, the sample set is divided according to the ratio of 7:3, where 70% of the samples are used as the training set for training samples and the remaining 30% are used as the test set for testing samples.

[0098] (3) Random Forest-based Intrusion Detection Model

[0099] Honeynet is designed as a system to attract, detect and analyze network attacks. Its core function is to accurately distinguish normal traffic from abnormal traffic. To achieve this goal, the random forest algorithm is introduced to build an intrusion detection model. This algorithm has excellent classification capabilities and can effectively judge the traffic flowing through the honeynet in real time. When abnormal traffic is captured, the honeynet system can respond in time, thereby effectively improving the network security protection capabilities of the industrial control system. The basic process of constructing the random forest is as follows: Figure 3 As shown, it specifically includes:

[0100] S1: Get sample data in the training set;

[0101] S2: Annotate the sample data;

[0102] S3: Encode the labels in the data;

[0103] S4: Train the initial random forest model on the training set, update the model parameters, and optimize the model;

[0104] S5: Test the trained random forest model on the test set;

[0105] S6: Build a random forest model.

[0106] (4) Dynamic Honeynet Strategy Optimization Based on DQN

[0107] As an important branch of reinforcement learning, DQN has the ability to handle complex environments and continuous decision-making processes. In order to build a dynamic honeynet based on real-time traffic that can respond to environmental changes, DQN can be used to dynamically adjust the optimal strategy of the honeynet.

[0108] On the basis of the trained random forest model, the traffic data detected by the random forest model is input into the reinforcement learning model, that is, the output of the random forest training model is used as the input of the reinforcement learning model. The reinforcement learning model determines the label in the traffic data. If the label is "BENIGN", the input traffic is normal traffic. Otherwise, it is abnormal traffic (port scanning type) data. At this time, DQN will perform abnormal traffic actions, such as modifying the honeypot fingerprint information, and obtain corresponding positive rewards. If the abnormal traffic action fails to be successfully executed, a corresponding negative reward will be obtained as a penalty. The abnormal traffic action is the configured honeynet strategy, and the dynamic configuration of the honeynet strategy is achieved through DQN. Its basic process is as follows: Figure 4 shown.

[0109] Through DQN reinforcement learning, in actual traffic data detection, the honeypot node makes inferences and predictions about traffic based on the characteristics of known data, and dynamically executes the configured strategy when capturing abnormal data.

[0110] Example

[0111] In order to verify the performance of the threat trapping method proposed in the present invention, the following simulation experiments were conducted.

[0112] 1. Experimental environment

[0113] The honeypot target machine is an Ubuntu 24.04 system based on the Linux 64-bit operating system, and the attack machine is a Kali 6.6.9 system based on the Debian 12.x 64-bit Linux distribution operating system. The machine learning hardware environment GPU is NVIDIA GeForce GTX 1650, and the software environment Python version is 3.11.9.

[0114] 2. Evaluation Metrics

[0115] The performance of the classification-based detection model is evaluated based on the number of abnormal traffic flows detected and the number of normal traffic flows. This embodiment evaluates the performance by calculating the accuracy index, which reflects the proportion of correctly classified samples. The calculation formula for Accuracy is:

[0116]

[0117] TP (True Positive) indicates the number of attacks correctly predicted, FP (False Positive) indicates the number of normal traffic records mistakenly predicted as attacks, TN (True Negative) indicates the number of normal traffic correctly predicted, and FN (False Negative) indicates the number of attacks mistakenly predicted as normal.

[0118] 3. Experimental analysis

[0119] Through the classification training of the random forest model, we can see that the evaluation of the performance index of the random forest model after training is 99.93%, and the accuracy of the prediction of the test set is 99.71%, both of which are very close to 1, such as Figure 5 shown.

[0120] Using the confusion matrix, we can intuitively see the performance of the model on different categories, such as Figure 6As shown in the figure, the model generally performs well in identifying each type of attack. However, its accuracy for "Infiltration" attacks is only around 80%, with a 20% probability of misclassifying it as a "Heartbleed" attack. For "Port Scan" attacks, there is also a 6.5% probability of misclassifying it as normal traffic. This is due to the imbalanced distribution of the dataset classes. The small number of examples for "Infiltration" and "Heartbleed" attacks prevents the model from fully learning the unique characteristics of these two attack types, resulting in insufficient recognition of these two types. For port scan attacks, the feature values ​​of the number of bytes transmitted and the source and destination port numbers are similar to those of normal traffic. Therefore, the model may misclassify some normal traffic as port scan attacks.

[0121] To address the 20% false positive rate of the random forest model for "Infiltration" attacks, we extracted all the data from the "Infiltration" and "Heartbleed" classes from the sample set into a new dataset and randomly sorted the dataset. To improve the model's accuracy in detecting these two types of attacks, we used Scikit-Learn's RandomizedSearchCV to find the best hyperparameter combination, optimized the model, and evaluated all combinations of hyperparameter values ​​through cross-validation. Finally, we selected the combination with the strongest generalization ability as the final hyperparameter for the model. The confusion matrix of the optimized model for detecting "Infiltration" and "Heartbleed" attacks is shown below. Figure 10 As we can see, after the random forest model's grid parameters were optimized, its detection accuracy for both "Infiltration" and "Heartbleed" attacks reached 100%, effectively resolving the initial model's high false positive rate for "Infiltration" attacks and improving system performance.

[0122] In order to analyze the relationship between the accuracy of the machine learning model and the total number of training samples, we extracted sample sets of different proportions for discussion, and expressed the accuracy corresponding to the sample sets of different proportions in the form of curves, as shown in the following figure: Figure 7 shown.

[0123] As you can see, the model's accuracy improves rapidly when the sample size is low, but as the sample size increases, the rate of improvement slows and stabilizes. Therefore, choosing a smaller sample set within the range where accuracy stabilizes can significantly reduce the time and cost of machine learning training while maintaining accuracy.

[0124] Figure 8 The graph shows that during the training iterations, the agent's reward growth curve is relatively steep at the beginning. After the fourth round of training, the growth rate begins to flatten out and fluctuates around the maximum value. This is because in the early stages of training, the neural network quickly learns from the most basic behaviors, resulting in a rapid increase in the Q value. Simultaneously, the ε value gradually decays to 0. Although this is an exponential decay process, the function curve in the figure visually resembles a logarithmic function. This is because the exponential decay rate is initially high but gradually slows with increasing iterations, resulting in a decrease in the slope of the curve, which takes on a logarithmic shape. This also demonstrates the agent's continuous transition from random exploration in the early stages to strategic utilization in the later stages.

[0125] The Nmap scanning tool on the Kali attack machine was used to identify the fingerprint information of Conpot using the info.nse script. After the honeynet detected the port scanning attack in the data capture module, it responded dynamically and modified the fingerprint information adaptively. The fingerprint information displayed after the scan result was "S C-C2M347785365". Figure 9 shown.

[0126] The final results show that the dynamic honeynet technology based on machine learning proposed in this invention can correctly judge abnormal traffic data and adaptively adjust the dynamic configuration of the honeynet strategy.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. The industrial control system threat trapping method based on machine learning algorithm is characterized by: The following steps are involved: Step 1: Modify the initial configuration file of the Conpot honeypot based on the characteristics of each device and protocol in the industrial control system; Step 2: Deploy multiple honeypot nodes of different types in the industrial control network to simulate different types of industrial control equipment; Step 3: Build a decoy model on each honeypot node; Step 4: Use the trapping model to judge the data traffic entering the honeypot node. When abnormal data is captured, the configured policy is dynamically executed. The specific steps of step 3 to establish the trapping model include: Step 3.1: Based on the CIC-IDS-2017 dataset, construct a traffic classification sample set; Step 3.2: Build a random forest model and train and test the model using the traffic classification sample set; Step 3.3: Use the output of the random forest model as the input of the DQN model; Step 3.4: The DQN model detects the input traffic data label, obtains different rewards based on the label type, calculates the target Q-value function based on the reward, optimizes the DQN model parameters by minimizing the loss function, and constructs a threat trapping model; Furthermore, the specific steps of step 3.4 include: Step 3.4.1: Obtain traffic data after random forest model detection; Step 3.4.2: The DQN model determines the label in the traffic data. If the label is "BENIGN", the current traffic data is considered abnormal and the process goes to step 3.4.

3. Otherwise, the current traffic data is normal and a negative reward is obtained. Step 3.4.3: Execute the abnormal traffic action. If the execution is successful, you will get the corresponding positive reward; otherwise, you will get the corresponding negative reward; Step 3.4.4: Calculate the target Q value based on the reward value, minimize the loss function, and determine whether the loss value of the loss function is less than or equal to the preset threshold. If not, use the gradient descent method to update the DQN parameters; if so, output the constructed DQN model.

2. The industrial control system threat trapping method based on machine learning algorithm according to claim 1, characterized in that: The specific steps of step 3.1 include: Step 3.1.1: Delete the feature values ​​of the two columns of stream byte rate and stream packet rate in the original data set data. NaN and Infinite Dirty data; Step 3.1.2: Randomly select one-third of the benign traffic data as the normal traffic data sample; Step 3.1.3: Filter out non-benign traffic from the remaining data in the original dataset as abnormal traffic samples; Step 3.1.4: Merge the normal traffic data samples and abnormal traffic samples and remove all dirty data rows to obtain the traffic classification sample set.

3. The industrial control system threat trapping method based on machine learning algorithm according to claim 1, characterized in that: The specific steps of step 3.2 include: Step 3.2.1: Standardize the data in the traffic classification sample set according to the following formula: in, Represents the original data, represents the data after normalization. represents the mean of the original data, represents its standard deviation; Step 3.2.2: Divide the normalized traffic classification sample set into a training set and a test set in a ratio of 7:3; Step 3.2.3: Encode the labels of the sample data in the training set and test set; Step 3.2.4: Train the random forest model on the training set, update the model parameters, and optimize the model; Step 3.2.5: Test the trained random forest model on the test set; Step 3.2.6: Build a random forest model.

4. The industrial control system threat trapping method based on machine learning algorithm according to claim 3, characterized in that: Step 3.4.4: Calculate the target Q value as follows: , calculate and modify the honeypot fingerprint information action through the following formula The corresponding Q value: in, γ Represents a discount factor that balances the weight between current rewards and potential future rewards; θ － Represents the parameters of the target network; r i Indicates that the agent is in state s Execute an action a After that, you will get the reward immediately.

5. The industrial control system threat trapping method based on machine learning algorithm according to claim 4, characterized in that: The loss function is: in, s Indicates a specific state. a Indicates the execution of an action. θ Represents the parameters of the DQN neural network, and N represents the number of samples.

Images