Industrial control system threat trapping method based on machine learning algorithm

Abstract

The invention discloses an industrial control system threat trapping method based on a machine learning algorithm, which aims at solving the problems of complicated abnormal traffic type, high feature dimension and the like of an electric power industrial control system, introduces a random forest algorithm to design a data detection module of a dynamic honeynet system, and improves the accuracy of abnormal traffic identification. And meanwhile, a DQN algorithm of deep reinforcement learning is utilized to help the honeynet system to make strategy dynamic adjustment when facing different attack behaviors. Experimental results show that the network security protection capability of the industrial control system can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of risk assessment, and in particular to an industrial control system threat trapping method based on a machine learning algorithm. Background Art

[0002] As a key infrastructure of modern society, the security of industrial control systems is of vital importance. However, with the evolution of the Industry 4.0 era, the new generation of information technology and industrialization are experiencing an unprecedented integration, which directly promotes the transformation of traditional closed industrial control systems towards openness and intelligence, and also makes them increasingly become the main target of cyber attacks.

[0003] With the rapid development of global network informatization, network security threats are constantly evolving and attack methods are becoming increasingly complex. The effectiveness of traditional network security protection measures is being severely tested in the face of ever-changing attack methods, especially those advanced persistent threats that exploit system vulnerabilities and unknown attack modes. In order to change the passive situation of the defender in cyberspace confrontation, security personnel have begun to focus on honeynet security defense technology based on active defense, which deploys honeypots and honeynets that simulate the characteristics of industrial control systems to lure attackers, capture attack behaviors and conduct research and analysis, and update defense strategies to strengthen the security protection capabilities of industrial control systems.

[0004] Regarding the honeynet technology for industrial control system security, Wang Yuzhen and others designed and implemented a smart grid honeypot framework based on reinforcement learning, SGPot, which can accurately simulate the system constant parameters in real equipment in the power industry to highly restore the control end of the smart substation. Wu Nanxu and others proposed an active defense solution combining K-means algorithm and honeypot technology to address the problems of high probability of successful intrusion and short intrusion time in current defense methods when resisting network attacks. Lin Zhi and others designed a honeypot framework based on industrial control equipment simulation for the high-risk functions of Modbus protocol and its affiliated UMAS protocol in industrial control systems to accurately simulate UMAS protocol messages and improve the trapping ability of honeypots. G.Wagener defined the configuration and behavior of high-interaction honeypots based on game theory concepts. As a variant of reinforcement learning, this method aims to help honeypots take the best defensive behavior when facing network attacks. Pauna and others first developed an adaptive medium-interaction honeypot system called RASSH, which can simulate SSH servers and use machine learning algorithms to interact intelligently with attackers. Yamamoto designed and implemented a framework for automatically generating IoT honeypots using FirmPot firmware. Seamus Dowling proposed an adaptive honeypot based on machine learning, which aims to respond to attack commands intelligently through learning. TOUCH et al. innovatively proposed the reinforcement learning honeypot architecture Asgard and its derivative architecture Midgard. LOPEZY et al. significantly improved the attacker's interactive participation in the adaptive honeypot environment by combining the word2vec model and deep reinforcement learning technology.

[0005] However, the static characteristics of the above traditional honeynet technology make it limited when dealing with complex network environments. Most honeypots are static defense methods that require manual deployment and configuration, and there are still problems with resource optimization and efficiency. Therefore, there is an urgent need for a more efficient strategy to reduce resource waste and improve the threat trapping model of system performance. Summary of the invention

[0006] In view of the shortcomings of the above-mentioned prior art, the present invention provides an industrial control system threat trapping method based on a machine learning algorithm, which comprises the following steps:

[0007] Step 1: Modify the initial configuration file of Conpot honeypot according to the characteristics of each device and protocol in the industrial control system;

[0008] Step 2: Deploy multiple honeypot nodes of different types in the industrial control network to simulate different types of industrial control equipment;

[0009] Step 3: Build a trapping model on each honeypot node;

[0010] Step 4: Use the trapping model to judge the data flow entering the honeypot node, and dynamically execute the configured strategy when abnormal data is captured.

[0011] Furthermore, the specific steps of step 3 to establish the trapping model include:

[0012] Step 3.1: Based on the CIC-IDS-2017 dataset, construct a traffic classification sample set;

[0013] Step 3.2: Build a random forest model and use the traffic classification sample set to train and test the model;

[0014] Step 3.3: Use the output of the random forest model as the input of the DQN model;

[0015] Step 3.4: The DQN model detects the input traffic data label, obtains different rewards according to the label type, calculates the target Q value function according to the reward, optimizes the DQN model parameters by minimizing the loss function, and constructs a threat trapping model.

[0016] Furthermore, the specific steps of step 3.1 include:

[0017] Step 3.1.1: Delete the dirty data with NaN and Infinite feature values ​​in the two columns of stream byte rate and stream packet rate in the original data set data;

[0018] Step 3.1.2: Randomly select one-third of the data from the benign traffic data as the normal traffic data sample;

[0019] Step 3.1.3: Filter out non-benign traffic from the remaining data of the original data set as abnormal traffic samples;

[0020] Step 3.1.4: Merge the normal traffic data samples and the abnormal traffic samples, and remove all dirty data rows to obtain the traffic classification sample set.

[0021] Furthermore, the specific steps of step 3.2 include:

[0022] Step 3.2.1: Standardize the data in the traffic classification sample set according to the following formula:

[0023]

[0024] Among them, x i represents the original data, z i represents the data after standardization, μ represents the mean of the original data, and σ represents its standard deviation;

[0025] Step 3.2.2: Divide the standardized traffic classification sample set into a training set and a test set in a ratio of 7:3;

[0026] Step 3.2.3: Encode the labels of the sample data in the training set and the test set;

[0027] Step 3.2.4: Train the random forest model on the training set, update the model parameters, and optimize the model;

[0028] Step 3.2.5: Test the trained random forest model on the test set;

[0029] Step 3.2.6: Build a random forest model.

[0030] Furthermore, the specific steps of step 3.4 include:

[0031] Step 3.4.1: Obtain the traffic data after random forest model detection;

[0032] Step 3.4.2: The DQN model determines the label in the traffic data. If the label is "BENIGN", the current traffic data is judged to be abnormal traffic and goes to step 3.4.3; otherwise, the current traffic data is normal traffic and a corresponding negative reward is obtained;

[0033] Step 3.4.3: Execute the abnormal traffic action. If the execution is successful, you will get the corresponding positive reward; otherwise, you will get the corresponding negative reward;

[0034] Step 3.4.4: Calculate the target Q value based on the reward value, minimize the loss function, and determine whether the loss value of the loss function is less than or equal to the preset threshold. If not, use the gradient descent method to update the DQN parameters; if so, output the constructed DQN model.

[0035] Furthermore, the step of calculating the target Q value in step 3.4.4 is as follows: according to the state s′ corresponding to the abnormal traffic data, the Q value corresponding to the action a′ of modifying the honeypot fingerprint information is calculated by the following formula:

[0036]

[0037] Among them, γ represents the discount factor, which is used to balance the weight between current rewards and future potential rewards; θ － Represents the parameters of the target network; r i It represents the immediate reward obtained by the agent after executing action a in state s.

[0038] Furthermore, the loss function is:

[0039]

[0040] Among them, s represents a specific state, a represents the execution action, θ represents the parameters of the DQN neural network, and N represents the number of samples.

[0041] Therefore, the present invention adopts the above-mentioned industrial control system threat trapping method based on machine learning algorithm, which has the following beneficial effects:

[0042] The present invention proposes a threat trapping technology for industrial control systems based on machine learning. An industrial control system simulation environment is built on Conpot. Real-time traffic detection and classification are performed based on the random forest model. The CIC-IDS-2017 dataset is used to train the random forest algorithm model to improve the accuracy of abnormal traffic identification. The DQN algorithm is introduced to realize the dynamic configuration of honeynet strategies, which effectively solves the problem of lack of dynamics in traditional honeynets.

[0043] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is the schematic diagram of the random forest algorithm.

[0045] Figure 2 This is the basic structure diagram of DQN.

[0046] Figure 3 This is the basic processing flow chart of random forest.

[0047] Figure 4 This is the basic flow chart of DQN.

[0048] Figure 5 It is the prediction result diagram in the embodiment.

[0049] Figure 6 It is a confusion matrix diagram of the prediction results in the embodiment.

[0050] Figure 7 It is a graph of accuracy rates corresponding to sample sets of different proportions in the embodiment;

[0051] Figure 8 (a)-(b) are the relationship diagrams of rewards and ε and training iterations during the training iteration process;

[0052] Fig. 9 This is the Nmap fingerprint scan image.

[0053] Fig.10 This is the confusion matrix of the optimized model for “Infiltration” and “Heartbleed” attack detection in the embodiment. DETAILED DESCRIPTION

[0054] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art make various changes or modifications to the present invention, and these equivalent forms also fall within the scope limited by the claims attached to the application.

[0055] The present invention proposes a threat trapping method for industrial control systems based on machine learning. It combines typical machine learning algorithms to construct a threat trapping model for industrial control systems composed of multiple different honeypots. The threat trapping model is used to solve the problem of lack of dynamics in traditional honeynets.

[0056] Next, the basic theory of building this model is introduced.

[0057] 1. Random Forest Algorithm

[0058] Random forest is a supervised ensemble model of the bagging type based on multiple independent decision trees. During the training process, k samples are extracted from the original training set N with replacement using the Bootstrap method to form k independent training sets, and then k decision trees are constructed. For each decision tree, a small number of features are randomly selected from all features for optimal segmentation. In the process of judging the input data, each decision tree will make a judgment to form a classification result. According to the principle of minority obeys majority, the one with the most classification results is the final output judgment result. Its basic principle is as follows: Figure 1 shown.

[0059] 2. DQN algorithm

[0060] The DQN (Deep Q-Network) algorithm is a reinforcement learning algorithm based on deep learning. Its core idea is the integration of deep learning and Q-learning. DQN is not only suitable for high-dimensional, continuous state space, but also can efficiently learn and make decisions in unknown environments, providing effective solutions to complex problems.

[0061] DQN's intelligent system uses deep neural networks to learn the state value function Q(s,a) in a complex environment, where s represents the state of the environment observed by the agent and a represents the action performed by the agent in that state. The Q-value function is an evaluation indicator used to measure the expected cumulative return that the agent can obtain by choosing to perform action a in a specific state s under a specific strategy π. DQN uses a neural network to approximate this function, as shown in the following formula:

[0062] Q(s, a; θ)≈Q * (s, a) (1)

[0063] Among them, Q*(s,a) represents the Q value of the optimal strategy, Q(s,a;θ) represents the approximate Q value output by the neural network, and θ represents the parameters of the neural network;

[0064] By using the target network to calculate the target value, the stability of learning is increased, and the target value is defined as follows:

[0065]

[0066] Among them, s′ represents the next state to be entered at the next moment; a′ represents the action at the next moment; γ represents the discount factor, which is used to balance the weight between the current reward and the potential reward in the future; θ － Represents the parameters of the target network; r i It represents the immediate reward obtained by the agent after executing action a in state s; the target value represents y i The maximum expected reward that can be obtained by taking the best action in state s′.

[0067] DQN will also establish a loss function, which is the mean square error (MSE) between the target value and the predicted Q value. This loss function is used to evaluate the deviation between the predicted Q value and the target Q value. The core goal of machine learning training is to minimize this deviation. The loss function is:

[0068]

[0069] Where N represents the number of samples;

[0070] Finally, in order to minimize the loss function, the gradient descent method is used to update the parameters θ of the neural network. The parameter update rule is as follows:

[0071]

[0072] Among them, α is the learning rate, is the gradient of the loss function with respect to the parameter θ.

[0073] In DQN, the environment is viewed as a dynamic and complex system, and the agent needs to continuously interact with it to collect experience data. This experience data includes the state observed by the agent, the actions performed, the immediate rewards obtained, and the next state. DNN uses this data to learn and approximate the Q value function and continuously optimize the network parameters by minimizing the prediction error. The entire structure of DQN is as follows Figure 2 shown.

[0074] 3. Build an industrial control dynamic honeynet system based on machine learning

[0075] (1) Conpot Honeypot

[0076] Conpot is an open source low-interaction industrial control honeypot developed and designed by the Mushroom Security team. It can establish a simulated industrial control system environment by simulating industrial control devices such as programmable logic controllers and SCADA servers and industrial control protocols such as S7comm, Modbus, and SNMP. The Conpot honeypot adopts a flexible and modular design, allowing users to customize the configuration based on the general configuration and simulate different scenarios by adding customized templates or protocols. Table 1 shows the core modules of the Conpot honeypot and its functions.

[0077] Table 1 Conpot modules and functions

[0078]

[0079] In the default template of Conpot, the template.xml file simulates the Siemens S7-200 series PLC. This file not only simulates the copyright information of the device, but also imitates the CPU information of the device. However, due to the defect of having Conpot's unique fingerprint information "88111222", it is easy to be identified by attackers and marked as "honeypot".

[0080] Therefore, according to the characteristics of the equipment and protocols in the industrial control system, the initial configuration file of the Conpot honeypot is modified, including custom addition or reduction of keys, or custom settings according to needs. Taking the Siemens S7-300 PLC device in the microgrid as an example, Conpot is simulated and configured. Table 2 lists the modified key parameter information in detail.

[0081] Table 2 Siemens S7-300 simulation parameter information

[0082]

[0083] A single honeypot can only simulate a specific device or service in an industrial control system, which cannot attract attackers. Therefore, in order to simulate a more realistic and attractive industrial control network environment, it is necessary to deploy multiple different types of honeypot nodes at the same time, simulate different types of industrial control equipment, and enable communication services to interact between honeypot nodes. In this way, when an attacker detects a honeypot node, he will also detect the data flow exchange with other honeypot nodes, and cannot determine whether it is a trap, thereby effectively enhancing the system's deception ability.

[0084] (2) CIC-IDS-2017 dataset

[0085] The CIC-IDS-2017 dataset covers 14 common attacks including Web attacks, penetration tests, brute force attacks, DDoS attacks, and port scans. The dataset consists of 8 files stored in csv format, as shown in Table 3. Each file records the complete network traffic data including 78 feature columns and one label column.

[0086] Table 3 CIC-IDS-2017 dataset files

[0087]

[0088] ①Data preprocessing

[0089] When constructing the traffic classification sample set, we first cleaned up some dirty data with feature values ​​of "NaN" and "Infinite" in the "Flow Bytes / s" and "Flow Packets / s" columns. Then, we randomly selected one-third of the data on "Monday" which was all benign traffic as normal traffic samples, and selected non-benign traffic from other files as abnormal traffic samples. Finally, we merged these two types of samples, removed all dirty data rows, and formed a complete sample set for analysis and model training. The number of samples with different labels in the traffic classification sample set is shown in Table 4.

[0090] Table 4 Number of samples for each label

[0091]

[0092] ② Standardization

[0093] Since the traffic classification sample set is large, standardizing the data to make it standard normal distribution (mean is 0, standard deviation is 1) can effectively reduce the training time of the machine learning model. The formula for standardizing the data is:

[0094]

[0095] Among them, x i represents the original data, z i represents the data after standardized processing, μ represents the mean of the original data, and σ represents its standard deviation.

[0096] ③ Sample set division

[0097] After completing the above two steps, the sample set is divided according to the ratio of 7:3, where 70% of the samples are used as the training set for training samples and the remaining 30% are used as the test set for testing samples.

[0098] (3) Intrusion Detection Model Based on Random Forest

[0099] Honeynet is designed as a system to attract, detect and analyze network attacks. Its core function is to accurately distinguish normal traffic from abnormal traffic. In order to achieve this goal, the random forest algorithm is introduced to build an intrusion detection model. The algorithm has excellent classification capabilities and can effectively judge the traffic flowing through the honeynet in real time. When abnormal traffic is captured, the honeynet system can respond in time, thereby effectively improving the network security protection capabilities of the industrial control system. The basic process of constructing a random forest is as follows: Figure 3 As shown, it specifically includes:

[0100] S1: Get sample data in the training set;

[0101] S2: Annotate the sample data;

[0102] S3: Encode the labels in the data;

[0103] S4: Train the initial random forest model on the training set, update the model parameters, and optimize the model;

[0104] S5: Test the trained random forest model on the test set;

[0105] S6: Build a random forest model.

[0106] (4) Dynamic Honeynet Strategy Optimization Based on DQN

[0107] As an important branch of reinforcement learning, DQN has the ability to handle complex environments and continuous decision-making processes. In order to build a dynamic honeynet based on real-time traffic that can respond to environmental changes, DQN can be used to dynamically adjust the optimal strategy of the honeynet.

[0108] Based on the trained random forest model, the traffic data detected by the random forest model is input into the reinforcement learning model, that is, the output of the random forest training model is used as the input of the reinforcement learning model. The reinforcement learning model determines the label in the traffic data. If the label is "BENIGN", the input traffic is normal traffic. Otherwise, it is abnormal traffic (port scanning type) data. At this time, DQN will perform abnormal traffic actions, such as modifying the honeypot fingerprint information, and obtain corresponding positive rewards. If the abnormal traffic action fails to be successfully executed, a corresponding negative reward will be obtained as a penalty. The abnormal traffic action is the configured honeynet strategy, and the dynamic configuration of the honeynet strategy is achieved through DQN. Its basic process is as follows: Figure 4 shown.

[0109] Through DQN reinforcement learning, in actual traffic data detection, the honeypot node infers and predicts the traffic based on the characteristics of known data, and dynamically executes the configured strategy when capturing abnormal data.

[0110] Example

[0111] In order to verify the performance of the threat trapping method proposed in the present invention, the following simulation experiment was carried out.

[0112] 1. Experimental environment

[0113] The honeypot target machine is an Ubuntu 24.04 system based on the Linux 64-bit operating system, and the attack machine is a Kali 6.6.9 system based on the Debian12.x 64-bit Linux distribution operating system. The machine learning hardware environment GPU is NVIDIA GeForce GTX 1650, and the software environment Python version is 3.11.9.

[0114] 2. Evaluation Metrics

[0115] The performance of the classification-based detection model is evaluated based on the number of abnormal traffic and the number of normal traffic detected. This embodiment evaluates by calculating the accuracy index Accuracy, which is used to reflect the proportion of correctly classified samples. The calculation formula of Accuracy is:

[0116]

[0117] Among them, TP (True Positive) indicates the number of attacks correctly predicted, FP (False Positive) indicates the number of normal traffic records mistakenly predicted as attacks, TN (True Negative) indicates the number of normal traffic correctly predicted, and FN (False Negative) indicates the number of attacks mistakenly predicted as normal.

[0118] 3. Experimental analysis

[0119] Through the classification training of the random forest model, we can see that the evaluation of the performance index of the random forest model after training is 99.93%, and the accuracy of the prediction of the test set is 99.71%, both of which are very close to 1, such as Figure 5 shown.

[0120] Using the confusion matrix, we can intuitively see the performance of the model on different categories, such as Figure 6As shown in the figure, we can see that the model performs well in judging each type of attack. However, in judging the "Infiltration" type of attack, the accuracy is only about 80%, and there is a 20% probability of misjudging it as a "Heartbleed" type of attack. For the "PortScan" type of attack, there is also a 6.5% probability of misjudging it as normal traffic. The reason is that the distribution of the data set categories is unbalanced. The number of samples of the "Infiltration" and "Heartbleed" types of attacks is small, which makes it impossible for the model to fully learn the unique features of these two types of attacks, resulting in insufficient recognition of the model for these two types of attacks. For port scanning attacks, the two columns of feature values, the number of transmitted bytes and the source and destination port numbers, are similar to normal traffic, so the model may misjudge some normal traffic as port scanning attacks.

[0121] In order to solve the problem that the random forest model has a false positive rate of 20% for "Infiltration" attacks, all the data of "Infiltration" and "Heartbleed" in the sample set are extracted into a new data set, and the data set is randomly sorted. In order to improve the accuracy of the model in detecting these two types of attacks, Scikit-Learn's RandomizedSearchCV is used to find the best hyperparameter combination, perform model optimization, and evaluate all combinations of hyperparameter values ​​through cross-validation. Finally, the combination with the strongest generalization ability is selected as the final hyperparameter of the model. The confusion matrix of the optimized model for "Infiltration" and "Heartbleed" attack detection is as follows: Fig.10 It can be seen that after the grid parameters of the random forest model were optimized, the accuracy of detecting the two types of attacks, "Infiltration" and "Heartbleed", has reached 100%, which effectively solved the problem of the high false positive rate of the initial model for the "Infiltration" type of attack detection and improved the system performance.

[0122] In order to analyze the relationship between the accuracy of the machine learning model and the total number of training samples, sample sets of different proportions were extracted for discussion, and the accuracy corresponding to sample sets of different proportions was expressed in the form of a curve, such as Figure 7 shown.

[0123] It can be seen that the accuracy of the model increases rapidly when the number of samples is low, but as the number of samples increases, the speed of accuracy improvement gradually slows down and tends to stabilize. Therefore, a smaller sample set can be selected within the range where the accuracy tends to stabilize, which greatly reduces the time and cost of machine learning training while ensuring accuracy.

[0124] Figure 8 It shows that during the training iteration, the growth curve of the agent's reward at the beginning is relatively steep, and after the fourth round of training, the growth rate begins to flatten and fluctuates around the maximum value. This is because in the early stages of training, the Q value increases rapidly due to the rapid learning of the neural network from the most basic behaviors. At the same time, the ε value gradually decays to 0. Although this is an exponential decay process, the function curve in the figure is visually similar to the curve of the logarithmic function. This is because the rate of exponential decay is very high at the beginning, but gradually slows down with the increase in the number of iterations, so that the slope of the curve decreases, presenting a shape similar to a logarithmic curve. This also shows that the agent has continuously transformed from random exploration in the early stage to strategy utilization in the later stage.

[0125] Through the Nmap scanning tool on the Kali attack machine, the info.nse script is used to identify the fingerprint information of Conpot. After the data capture module detects the port scanning attack, the honeynet has made a dynamic response and adaptively modified the fingerprint information. The fingerprint information displayed after the scan result is "S C-C2M347785365", as shown in the figure. Fig. 9 shown.

[0126] The final results show that the dynamic honeynet technology based on machine learning proposed in this invention can correctly judge abnormal traffic data and adaptively adjust the dynamic configuration of the honeynet strategy.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. The industrial control system threat trapping method based on machine learning algorithm is characterized by: The following steps are involved: Step 1: Modify the initial configuration file of Conpot honeypot according to the characteristics of each device and protocol in the industrial control system; Step 2: Deploy multiple honeypot nodes of different types in the industrial control network to simulate different types of industrial control equipment; Step 3: Build a trapping model on each honeypot node; Step 4: Use the trapping model to judge the data flow entering the honeypot node, and dynamically execute the configured strategy when abnormal data is captured.

2. The industrial control system threat trapping method based on machine learning algorithm according to claim 1, characterized in that: Step 3: The specific steps of establishing the trapping model include: Step 3.1: Based on the CIC-IDS-2017 dataset, construct a traffic classification sample set; Step 3.2: Build a random forest model and use the traffic classification sample set to train and test the model; Step 3.3: Use the output of the random forest model as the input of the DQN model; Step 3.4: The DQN model detects the input traffic data label, obtains different rewards according to the label type, calculates the target Q value function according to the reward, optimizes the DQN model parameters by minimizing the loss function, and constructs a threat trapping model.

3. The industrial control system threat trapping method based on machine learning algorithm according to claim 1, characterized in that: The specific steps of step 3.1 include: Step 3.1.1: Delete the dirty data with NaN and Infinite feature values ​​in the two columns of stream byte rate and stream packet rate in the original data set data; Step 3.1.2: Randomly select one-third of the data from the benign traffic data as the normal traffic data sample; Step 3.1.3: Filter out non-benign traffic from the remaining data of the original data set as abnormal traffic samples; Step 3.1.4: Merge the normal traffic data samples and the abnormal traffic samples, and remove all dirty data rows to obtain the traffic classification sample set.

4. The industrial control system threat trapping method based on machine learning algorithm as claimed in claim 3, characterized in that: The specific steps of step 3.2 include: Step 3.2.1: Standardize the data in the traffic classification sample set according to the following formula: Among them, x i represents the original data, z i represents the data after standardization, μ represents the mean of the original data, and σ represents its standard deviation; Step 3.2.2: Divide the standardized traffic classification sample set into a training set and a test set in a ratio of 7:3; Step 3.2.3: Encode the labels of the sample data in the training set and the test set; Step 3.2.4: Train the random forest model on the training set, update the model parameters, and optimize the model; Step 3.2.5: Test the trained random forest model on the test set; Step 3.2.6: Build a random forest model.

5. The industrial control system threat trapping method based on machine learning algorithm as claimed in claim 2, characterized in that: The specific steps of step 3.4 include: Step 3.4.1: Obtain the traffic data after random forest model detection; Step 3.4.2: The DQN model determines the label in the traffic data. If the label is "BENIGN", the current traffic data is judged to be abnormal traffic and goes to step 3.4.3; otherwise, the current traffic data is normal traffic and a corresponding negative reward is obtained; Step 3.4.3: Execute the abnormal traffic action. If the execution is successful, you will get the corresponding positive reward; otherwise, you will get the corresponding negative reward; Step 3.4.4: Calculate the target Q value based on the reward value, minimize the loss function, and determine whether the loss value of the loss function is less than or equal to the preset threshold. If not, use the gradient descent method to update the DQN parameters; if so, output the constructed DQN model.

6. The industrial control system threat trapping method based on machine learning algorithm according to claim 5, characterized in that: Step 3.4.4 The steps for calculating the target Q value are: according to the state s' corresponding to the abnormal traffic data, the Q value corresponding to the action a' of modifying the honeypot fingerprint information is calculated by the following formula: Among them, γ represents the discount factor, which is used to balance the weight between current rewards and future potential rewards; θ － Represents the parameters of the target network; r i It represents the immediate reward obtained by the agent after executing action a in state s.

7. The industrial control system threat trapping method based on machine learning algorithm as claimed in claim 5, characterized in that: The loss function is: Among them, s represents a specific state, a represents the execution action, θ represents the parameters of the DQN neural network, and N represents the number of samples.

Images