Method for evaluating simulation capability of industrial control honey pot

Abstract

The invention provides an industrial control honey pot simulation capability evaluation method, and relates to the technical field of network security. The evaluation method comprises an evaluation method based on a BERT model, an evaluation method based on log data, an evaluation method based on interactive verification and an evaluation method based on memory fingerprints. Evaluating industrial protocol interaction by an evaluation method based on data packet fragmentation, a feature extraction evaluation method based on fuzzy testing, an evaluation method based on an IEC104 protocol and an evaluation method based on a successive probability discrimination algorithm; evaluating the mixed type data by a Bayesian network-based evaluation method, a deep learning-based integrated evaluation method and a machine learning-based evaluation method; and accumulating the scores obtained by all the evaluation methods and averaging the scores to obtain a honeypot simulation mass score. According to the method, each dimension of the industrial control honey pot is quantified into a specific score, and finally a comprehensive honey pot simulation quality score is given, so that the evaluation result is more accurate.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a method for evaluating the simulation capability of an industrial control honeypot. Background Art

[0002] An industrial control system (ICS) is a system used to control and manage industrial equipment, also known as an industrial control system (ICS). Programmable logic controllers (PLCs) are core devices specifically used to manage ICS. Managers interact with PLCs to control ICS equipment, obtain device status information, and control complex industrial processes.

[0003] With the increasing threat of attacks against industrial control systems, attackers attempt to manipulate PLCs to carry out malicious activities. These attacks are becoming increasingly complex and diverse, making industrial control honeypot technology indispensable. Honeypots are deployed on the public internet, disguised as PLC devices to lure attackers into attacking them. Attackers' attacks on the honeypots do not harm the actual industrial control system equipment. Industrial control honeypots record all interactions with attackers, enabling researchers to analyze the attacker's methods and subsequently improve and optimize the honeypot design. These honeypots can also be used for intelligent system-wide assessments when deploying them on a large scale. Consequently, industrial control honeypots are becoming an effective means of detecting new threats to industrial control systems.

[0004] However, the trapping capability of an industrial control honeypot depends on the completeness of its simulation of the operating status of PLC devices and the depth of its interaction with attackers. How to effectively evaluate the simulation capability of an industrial control honeypot has become a key issue. Existing technologies evaluate industrial control honeypots from different dimensions, with relatively single evaluation angles and one-sided evaluation standards. Therefore, there is an urgent need to provide a solution to improve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for evaluating the simulation capability of an industrial control honeypot, so as to solve the problem that the existing technology has a relatively single evaluation angle and a relatively one-sided evaluation standard when evaluating the simulation capability of an industrial control honeypot.

[0006] The present invention provides a method for evaluating the simulation capability of an industrial control honeypot, which adopts the following technical solutions:

[0007] The first evaluation score is obtained by evaluating the device system level based on the BERT model. The second evaluation score is obtained based on the log entry integrity, time series consistency, event and operation correlation, and log feature statistical calculation in the honeypot log. The third evaluation score is obtained by evaluating the device system level based on interactive verification. The fourth evaluation score is obtained by evaluating the device system level based on memory fingerprint.

[0008] Based on the data packet fragmentation evaluation method to simulate normal industrial control protocol requests, the honeypot was evaluated based on the four dimensions of the general method, Modbus protocol, S7comm protocol, and IEC104 protocol, and obtained the fifth evaluation score. The industrial protocol interaction was evaluated based on the feature extraction of fuzz testing and obtained the sixth evaluation score. The industrial protocol interaction was evaluated based on the IEC104 protocol and obtained the seventh evaluation score. The industrial protocol interaction was evaluated based on the successive probability discrimination algorithm and obtained the eighth evaluation score.

[0009] The evaluation of mixed-type data based on Bayesian networks received the ninth evaluation score, the evaluation of mixed-type data based on deep learning integration received the tenth evaluation score, and the evaluation of mixed-type data based on machine learning received the eleventh evaluation score.

[0010] The first to eleventh evaluation scores are accumulated and averaged to obtain a comprehensive honeypot simulation quality score.

[0011] Optionally, the process of evaluating the device system level based on the BERT model to obtain a first evaluation score includes:

[0012] Acquire industrial control equipment data based on network scanning and search engines, and pre-process and annotate the industrial control equipment data;

[0013] The BERT model is trained based on the labeled data to enable it to identify the characteristic entities of the industrial control honeypot. The trained BERT model is evaluated and optimized using the precision, recall, and F1 score indicators. The honeypot is evaluated based on the optimized BERT model to obtain the first evaluation score. The mathematical expression for the BERT model's evaluation of the honeypot is:

[0014] S=100×(w p P+w r ·R+w f F1);

[0015] Among them, S represents the BERT evaluation score, P represents the precision score, R represents the recall score, F1 represents the harmonic mean of precision and recall, and w p 、w r 、w f They represent the weights of precision, recall and comprehensive index F1 respectively.

[0016] Optionally, the process of evaluating the device system level based on the interactive verification to obtain a third evaluation score includes:

[0017] Filter trusted IP nodes based on credit information, exclude known proxy nodes, CDN nodes, and egress gateway nodes, scan the remaining IP nodes, match the features in the open source honeypot feature library, and filter out the IP of the target honeypot;

[0018] Perform detailed scans of target IP nodes, collect asset information, analyze the number of open ports and periodic changes in asset fingerprints, remotely log in and execute system commands, and obtain login success rates and command feedback results;

[0019] The third evaluation score is obtained by scoring the IP node based on the number of open ports, login success rate, and command feedback results.

[0020] Optionally, the process of evaluating the device system level based on the memory fingerprint to obtain the fourth evaluation score includes:

[0021] The SSM evaluation score is obtained based on the content structure similarity and semantic conflict of the built-in operating system, the CRM evaluation score is obtained based on the memory processing service and logic execution of the memory area, the IOM evaluation score is obtained based on sensor noise and industrial process dynamics, and the fourth evaluation score is calculated based on the SSM evaluation score, CRM evaluation score and IOM evaluation score.

[0022] Optionally, the process of evaluating the industrial protocol interaction based on feature extraction using fuzz testing to obtain a sixth evaluation score includes:

[0023] The effectiveness score is obtained based on the average number of response messages and the average response number threshold. The diversity score is obtained based on the number of different types of response messages received and the response type number threshold. The balance score is obtained based on the distribution variance of the number of different types of response messages and the variance threshold.

[0024] Weighting the effectiveness score, diversity score, and balance score to obtain a set of probe messages from the top m groups, filtering out a union of response message types corresponding to the probe messages from the set, and obtaining a coverage score based on the union and the number of all possible response message types;

[0025] The effectiveness score, diversity score, balance score, and coverage score are added together to obtain the sixth assessment score.

[0026] Optionally, the process of evaluating industrial protocol interaction based on the IEC 104 protocol to obtain the seventh evaluation score includes:

[0027] Based on the monitoring tool, IEC104 protocol data packets are captured from the industrial control system, the data packets are analyzed and the message structure and interaction logic are extracted. Based on the message format of the IEC104 protocol, a malformed detection data packet is constructed. The malformed detection data packet is used to test known devices and honeypots, and the responses are recorded. An abnormal feature library and a detection data packet library are constructed, and a detection is initiated on the target device. The response data packets are collected, and based on the degree of matching between the response data packet and the abnormal feature library, the honeypot is evaluated by comparing with the preset threshold to obtain the seventh evaluation score.

[0028] Optionally, the process of evaluating the industrial protocol interaction based on the successive probability discrimination algorithm to obtain the eighth evaluation score includes:

[0029] Gradually introduce features, calculate the conditional probability that the device is a honeypot under each feature, and compare it with the set threshold to decide whether to identify the device as a honeypot;

[0030] The conditional probability is calculated using the first feature. If it is greater than the set threshold, the device is considered a honeypot. If it is less than the threshold, the next feature is introduced and the conditional probability is calculated until all features are used or the conditional probability exceeds the threshold.

[0031] For devices running the protocol, the ratio of the conditional probability to the total number of samples is calculated to determine whether the device is a honeypot and obtain the eighth evaluation score.

[0032] Optionally, the process of evaluating the mixed-type data based on the Bayesian network to obtain the ninth evaluation score includes:

[0033] Build a Bayesian network structure model, determine the correspondence between honeypot features and model node states, learn model parameters, and obtain the first probability value of the honeypot;

[0034] Evaluate the performance of the Bayesian network model by comparing the ROC curves and AUC values of models generated by different machine learning algorithms;

[0035] Input feature evidence, use the Bayesian network inference algorithm to calculate the second probability value, and compare it with the first probability value. When the second probability value is greater than or equal to the first probability value, it is considered to be a honeypot, otherwise it is considered not to be a honeypot. The honeypot is evaluated to obtain the ninth evaluation score.

[0036] Optional, the process of evaluating mixed-type data based on deep learning integration to obtain the tenth evaluation score includes:

[0037] Identify honeypots and industrial control devices based on strong feature data. Use the common features and recognition results corresponding to the identified honeypots and industrial control devices as training sets, input them into the deep learning model for training, and build a honeypot recognition model.

[0038] The online feature recognition module extracts the first feature data from the original data of different types of industrial control equipment, inputs it into the honeypot recognition model to obtain the first recognition result with a probability value. If the recognition is wrong, the wrong result is corrected based on the model update module, and the corrected recognition result is used as a training set to input the offline honeypot recognition model for training. After the training is completed, it is reused for online recognition to obtain the tenth evaluation score.

[0039] Optionally, an eleventh evaluation score may be obtained by evaluating mixed-type data based on machine learning, including:

[0040] Calculate the probability that the target IP is an industrial control honeypot in each step, and assign a weight to each probability based on a machine learning method; calculate the comprehensive probability of the target IP by weighted summation, evaluate the honeypot based on the comprehensive probability, and obtain an eleventh evaluation score.

[0041] The present invention provides a method for evaluating the simulation capability of an industrial control honeypot, which has the following beneficial effects:

[0042] 1. This invention divides the industrial control honeypot simulation capability assessment into three assessment modules according to data type. It can quantify the assessment of each dimension of the industrial control honeypot into specific investigations and ultimately provide a comprehensive honeypot simulation quality score. This mitigates the problem of inconsistent evaluation results that is easily caused by most existing technologies that only investigate a single dimension, making the evaluation results more accurate.

[0043] 2. The present invention can quantify the simulation quality of the honeypot by calculating the honeypot simulation quality score, thereby providing a basis for optimizing the honeypot;

[0044] 3. The present invention can adjust the weight of each sub-item and the proportion of different evaluation methods in the total score through experimental verification to adapt to different application scenario requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of the evaluation method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the invention belongs. The words "including" and similar words used in this article mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0047] The embodiment of the present invention provides a method for evaluating the simulation capability of an industrial control honeypot. Figure 1 ,include:

[0048] S1. The first evaluation score is obtained by evaluating the device system level based on the BERT model. The second evaluation score is obtained based on the log entry integrity, time series consistency, event and operation correlation, and log feature statistical calculation in the honeypot log. The third evaluation score is obtained by evaluating the device system level based on interactive verification. The fourth evaluation score is obtained by evaluating the device system level based on memory fingerprinting.

[0049] S2. Based on the data packet fragmentation evaluation method, the normal industrial control protocol request is simulated. The honeypot is evaluated based on the four dimensions of the general method, Modbus protocol, S7comm protocol, and IEC104 protocol, and the fifth evaluation score is obtained. The industrial protocol interaction is evaluated based on the feature extraction of fuzzy testing and the sixth evaluation score is obtained. The industrial protocol interaction is evaluated based on the IEC104 protocol and the seventh evaluation score is obtained. The industrial protocol interaction is evaluated based on the successive probability discrimination algorithm and the eighth evaluation score is obtained.

[0050] S3: The ninth evaluation score was obtained by evaluating mixed-type data based on Bayesian networks, the tenth evaluation score was obtained by evaluating mixed-type data based on deep learning integration, and the eleventh evaluation score was obtained by evaluating mixed-type data based on machine learning.

[0051] S4. Accumulate the first to eleventh evaluation scores and take the average value to obtain a comprehensive honeypot simulation quality score.

[0052] In some embodiments, the process of executing step S1 includes:

[0053] S1.1. Evaluate the device system level based on the BERT model and obtain the first evaluation score;

[0054] S1.2. Obtain a second evaluation score based on the integrity of log entries, time series consistency, event-operation correlation, and log feature statistics in the honeypot logs;

[0055] S1.3. Evaluate the device system level based on interactive verification to obtain a third evaluation score;

[0056] S1.4. Evaluate the device system level based on the memory fingerprint to obtain a fourth evaluation score.

[0057] Specifically, the process of executing step S1.1 includes:

[0058] S1.1.1, data acquisition and preprocessing;

[0059] S1.1.2. Obtain the first assessment score.

[0060] Specifically, in executing step S1.1.1, data acquisition and preprocessing includes: acquiring industrial control equipment data based on network scanning and search engines, and preprocessing and labeling the industrial control equipment data.

[0061] In fact, the industrial control equipment data includes IP address, operating system, network traffic and other detailed information such as port, id, asn, timestamp, isp and name.

[0062] Furthermore, the BERT model is trained based on the labeled data to enable it to identify the characteristic entities of the industrial control honeypot. The trained BERT model is evaluated and optimized using the precision, recall, and F1 score indicators. The honeypot is evaluated based on the optimized BERT model to obtain the first evaluation score. The mathematical expression for the BERT model's evaluation of the honeypot is:

[0063] S=100×(w p P+w r ·R+w f F1);

[0064] Among them, S represents the BERT evaluation score, P represents the precision score, R represents the recall score, F1 represents the harmonic mean of precision and recall, and w p 、w r 、w f They represent the weights of precision, recall and comprehensive index F1 respectively.

[0065] Specifically, the process of executing step S1.2 includes:

[0066] S1.2.1. Obtain log entry completeness scores;

[0067] S1.2.2. Obtain time series consistency scores;

[0068] S1.2.3. Obtain the event-operation correlation score;

[0069] S1.2.4. Obtain log feature statistical scores;

[0070] S1.2.5. Obtain a second assessment score.

[0071] Specifically, the process of executing step S1.2.1 includes: obtaining industrial equipment log entries, including common records such as configuration changes, operating time, event alarms, etc. Using industrial control protocol log analysis tools (such as Wireshark log capture tool) to obtain operation logs from real PLCs, and performing comparative analysis on log content and density.

[0072] Furthermore, the log entry completeness score is calculated using the following formula:

[0073]

[0074] Specifically, the process of executing step S1.2.2 includes: obtaining a log timestamp sequence, including the time intervals and distribution patterns of log generation, performing time series analysis on the log timestamps, and using a model such as ARIMA or smooth curve fitting to detect anomalies in the time intervals.

[0075] Furthermore, the time series consistency score is calculated using the following formula:

[0076]

[0077] Specifically, the process of executing step S1.2.3 includes: obtaining records of operation events and related events in the log, including configuration changes and alarms triggered by the operation. Using log analysis tools (such as ELK Stack), the mapping relationship between events and operations is extracted to verify whether the honeypot log has similar logic.

[0078] Furthermore, the following formula is used to calculate the event-operation correlation score:

[0079]

[0080] Specifically, step S1.2.4 includes obtaining the distribution ratio of different types of logs, such as the proportion of alarm logs, configuration change logs, and status logs. Based on a large sample of real PLC logs, the characteristic distribution is statistically analyzed to compare and obtain the deviation between the honeypot logs and the real distribution.

[0081] Furthermore, the log feature statistical score is calculated using the following formula:

[0082]

[0083] Furthermore, in the process of executing step S1.2.5 and obtaining the second evaluation score, the maximum score of each dimension is 25 points, and the total score of the four dimensions is added together, with a maximum of 100 points.

[0084] Specifically, the process of executing step S1.3 includes:

[0085] S1.3.1、Acquisition of required data;

[0086] S1.3.2. Obtain the third assessment score.

[0087] Specifically, in the process of executing step S1.3.1, obtaining the required data includes:

[0088] Collect relevant information about IP nodes through public channels such as WHOIS queries, DNS queries, and search engines, and evaluate the credit of IP nodes in combination with historical data and reputation databases. Build a database containing features such as Dionaea, Conpot, kippo, and t-pot, and update it regularly to include the latest honeypot features. Use network scanning tools (such as Nmap) to perform port scans on IP nodes to collect open port information, service types, device names, technical protocols, etc. Try to remotely log in to the IP node through protocols such as SSH, FTP, Telnet, SMB, and record the number of successful login attempts and results. After a successful login, execute system operation commands (such as `history`, `ll`, `top`, etc.) and analyze the feedback results after the command execution.

[0089] Furthermore, step S1.3.2 is executed to filter trusted IP nodes based on the credit information, excluding known proxy nodes, CDN nodes, and egress gateway nodes. The remaining IP nodes are scanned and matched with the features in the open source honeypot feature library to filter out the IP of the target honeypot.

[0090] Furthermore, the target IP node is scanned in detail, asset information is collected, the number of open ports and periodic changes in asset fingerprints are analyzed, system commands are remotely logged in and executed, and the login success rate and command feedback results are obtained;

[0091] Specifically, when scoring the IP node based on the number of open ports, login success rate, and command feedback results to obtain the third evaluation score, the following formula is used:

[0092] S3=W1×S 端口 +W2×S 登录 +W3×S 命令反馈 +W4×S 资产信息 +W5×S 行为模式 ;

[0093]

[0094] S 登录 =100-(number of successful logins × 10);

[0095] S 命令反馈 =100-(number of abnormal command feedback times × 20);

[0096] S 资产信息 =100-(number of periodic changes × 15);

[0097] S 行为模式 =100-(abnormal behavior confidence × 100);

[0098] Among them, W1+W2+W3+W4+W5=1, and W1, W2, W3, W4, and W5 represent the interactive verification weights.

[0099] Specifically, the process of executing step S1.4 includes:

[0100] S1.4.1. Obtain an SSM evaluation score based on the content structure similarity and semantic conflict of the built-in operating system; S1.4.2. Obtain a CRM evaluation score based on the memory processing services and logic execution of the memory area;

[0101] S1.4.3. Obtain an IOM assessment score based on sensor noise and industrial process dynamics;

[0102] S1.4.4. Obtain the fourth assessment score.

[0103] Specifically, the process of executing step S1.4.1 includes: obtaining the memory structure by calculating the byte-level similarity of the SSM in the real PLC and the suspicious target, using the Levenshtein Distance algorithm to measure the similarity of the memory structure, and determining the semantic conflict by checking whether various semantic rules in the SSM are violated, such as the synchronization of the CPU status and the module indicator status, the automatic refresh of the interrupt status, etc. These rules are extracted from the PLC manual and verified on the real PLC.

[0104] Furthermore, the SSM is responsible for storing key status information related to the built-in operating system during PLC operation. The content is created only by the CPU and cannot be modified directly by the user. The high total score of 40 points highlights its importance. The memory structure similarity (λ1) score is included: the Levenshtein distance is used to quantify the byte-level differences between the SSM items of the suspected target and the real PLC, and the similarity Sb(i) of each overlapping item is calculated, and then the average is taken to obtain λ1. Let the threshold range of λ1 be [τ min ,τ max](obtained through multiple sampling statistics of real PLC), initial score

[0105] Furthermore, semantic conflict (λ2) scoring: There should be no semantic conflict in real PLC, so the semantic fields in the honeypot SSM are analyzed to check whether they violate the predefined semantic rules. For each semantic conflict found, 5 points are deducted. Let the number of semantic conflicts found be n, and the score S be SSM2 =40-5n. If n≥8, then S SSM2 =0, the SSM score is the average of the two scores, and the SSM calculation formula is: S SSM =0.5×S SSM1 +0.5×S SSM2 .

[0106] Specifically, during step S1.4.2, the aforementioned operations are executed via network requests, and the results are collected to evaluate the CRM processing service. Simultaneously, test control logic is generated and converted into C code, which is then executed on a desktop PC to remotely verify the target calculations and compare the variables in the test control logic with those in the digital twin program.

[0107] Furthermore, CRM, as a user-defined memory area, stores key information such as hardware configuration, control logic, and related constants and variables. It is the core element for PLC to achieve precise control of industrial processes. It determines the connection method and communication parameters between PLC and external devices, etc. Its high total score of 40 points highlights its importance.

[0108] Furthermore, the Memory Handling Service (λ3) score tests whether the suspected target CRM block can correctly handle create, read, overwrite, and delete operations. Any incomplete or incorrect implementation of an operation will increase the λ3 value. Each failed operation will be deducted 5 points. These operations are critical to the normal functioning of the CRM, and any failure of an operation will affect its authenticity. Let the number of failed operations be m, and the score be: S CRM1 =40-5m.

[0109] Furthermore, the logic execution (λ4) score is calculated by comparing the C code execution results of the test control logic with the actual execution results of the target honeypot, and the inconsistency ratio p of the variables in the test control logic and the digital twin program is calculated, and the score is S CRM2 =40×(1-p), the total CRM score is the average of the two scores, and the CRM calculation formula is: S CRM =0.5×S CRM1 +0.5×S CRM2 .

[0110] Specifically, during the execution of step S1.4.3, sensor data is continuously sampled from the target honeypot and the IOM of the real PLC, and the fitting probability distribution Y is constructed. d and It is used to evaluate the sensor noise distribution (λ5). It is used to evaluate the industrial process dynamics (λ6) by introducing control signal changes and comparing the response of the target honeypot with the ground truth of the historical dataset.

[0111] Furthermore, compared with the complex logic and strict memory management mechanisms in SSM and CRM, the simulation of sensor noise distribution and industrial process dynamics in IOM is relatively easy to implement, with relatively small discrimination, and the total score is set to 20 points.

[0112] Furthermore, the sensor noise distribution (λ5) score is calculated as follows: the sensor noise distribution Y of the suspected target is calculated as follows: d Noise distribution of the real process The Jensen-Shannon divergence λ5 between them is calculated. Then calculate λ5 and the ideal threshold ( The difference ratio is determined by analyzing the actual PLC process The scoring formula is S IOM1 =20×(1-q).

[0113] Furthermore, the industrial process dynamics (λ6) score is calculated by comparing the response of the target honeypot with the benchmark truth value of the historical data set, and then calculating the sequence residual λ6 and the ideal threshold The difference ratio (determined by analyzing the actual PLC process) Score: S IOM2 =20×(1-r), and the total IOM score is the average of the two scores. When λ5 and λ6 are closer to their respective ideal thresholds, the noise distribution and industrial process dynamics of the honeypot IOM are closer to the real PLC equipment, and the score is higher. The IOM calculation formula is: S IOM =0.5×S IOM1 +0.5×S IOM2 .

[0114] Specifically, during the execution of step S1.4.4, the test results of multiple memory features are integrated based on the HoneyJudge framework, and the final honeypot evaluation result is obtained through a weighted voting algorithm.

[0115] In some embodiments, the process of executing step S2 includes:

[0116] S2.1. Using the packet fragmentation evaluation method to simulate normal industrial control protocol requests, the honeypot was evaluated based on four dimensions: general method, Modbus protocol, S7comm protocol, and IEC104 protocol, achieving a fifth-place evaluation score.

[0117] S2.2. Evaluate industrial protocol interactions based on feature extraction using fuzzy testing to obtain a sixth evaluation score. S2.3. Evaluate industrial protocol interactions based on the IEC104 protocol to obtain a seventh evaluation score.

[0118] S2.4. Evaluate the industrial protocol interaction based on the successive probability discrimination algorithm to obtain an eighth evaluation score.

[0119] Specifically, the process of executing step S2.1 includes: sending a fragmented data packet and sending it again after a specific time interval (2n seconds). If the target system fails to correctly respond to the second data packet within the set time, a first response score is obtained; sending a fragmented Modbus protocol message with a length of less than n bytes. If the target system responds abnormally, it indicates that the Modbus protocol simulation is incomplete, and a second response score is obtained; analyzing the length of the COTP segment, fragmenting the COTP segment message multiple times, and if the target system responds abnormally, it indicates that the S7comm protocol simulation is incomplete, and a third response score is obtained; sending a specific message and setting a maximum waiting time. If the target system fails to make a valid response within the maximum waiting time, it indicates that the IEC104 protocol simulation is incomplete, and a fourth response score is obtained;

[0120] Furthermore, the first response score, the second response score, the third response score and the fourth response score are weighted to obtain a fifth evaluation score; wherein the total response score S general The mathematical expression is:

[0121]

[0122] Where t represents the waiting time.

[0123] Specifically, the process of executing step S2.2 includes: obtaining an effectiveness score based on the average number of response messages and the average response number threshold, obtaining a diversity score based on the number of different types of response messages received and the response type number threshold, and obtaining a balance score based on the distribution variance of the number of different types of response messages and the variance threshold;

[0124] Furthermore, the effectiveness score, diversity score, and balance score are weighted to obtain a set of probe messages from the top m groups in the ranking, a union of response message types corresponding to the probe messages is filtered out from the set, and a coverage score is obtained based on the union and the number of all possible response message types;

[0125] Furthermore, the effectiveness score, diversity score, balance score, and coverage score are added together to obtain a sixth evaluation score.

[0126] In fact, the calculation formula of the sixth evaluation score is:

[0127] S6=S E +S D +S B +S C ;

[0128]

[0129]

[0130] Among them, S6 represents the coverage score, S E Score for effectiveness, S D S is the diversity score, B is the balance score, S C is the coverage score, Indicates the average number of response messages, R max represents the average response number threshold, σ 2 represents the variance, represents the variance threshold, U represents the number of all possible response message types, and U max Indicates the number threshold of message types, t indicates the waiting time, T max Indicates the response type quantity threshold.

[0131] Specifically, in the process of executing step S2.3, it includes: capturing IEC104 protocol data packets from the industrial control system based on the monitoring tool, analyzing the data packets and extracting the message structure and interaction logic, constructing a deformed detection data packet based on the message format of the IEC104 protocol, using the deformed detection data packet to test known devices and honeypots, recording responses, building an abnormal feature library and a detection data packet library, initiating detection on the target device, collecting response data packets, and evaluating the honeypot based on the degree of matching between the response data packet and the abnormal feature library by comparing with a preset threshold to obtain the seventh evaluation score.

[0132] Furthermore, the seventh evaluation score S7 is calculated as follows:

[0133]

[0134] Among them, R match is the matching ratio between the normal response and the feature library, ranging from [0,1]. The higher the matching, the more inclined to the honeypot. var is the diversity score of the measured data packet, ranging from [0,1]. The more diverse the data packet, the better the simulation. response ),σ(T response) are the mean and volatility of response time, A freq is the abnormal response frequency, ranging from [0,1]. The lower it is, the closer it is to the real device.

[0135] Specifically, the process of executing step S2.4 includes: gradually introducing features, calculating the conditional probability that the device is a honeypot under each feature, and comparing it with the set threshold to decide whether to identify the device as a honeypot; using the first feature to calculate the conditional probability, if it is greater than the set threshold, the device is considered to be a honeypot; if it is less than the threshold, continue to introduce the next feature and calculate the conditional probability until all features are used or the conditional probability exceeds the threshold; for devices running the protocol, determine whether the device is a honeypot by calculating the ratio of the conditional probability to the total number of samples, and obtain the eighth evaluation score.

[0136] In some embodiments, the process of executing step S3 includes:

[0137] S3.1. Evaluate mixed-type data based on the Bayesian network to obtain the ninth evaluation score;

[0138] S3.2, evaluate mixed-type data based on deep learning integration and obtain the tenth evaluation score;

[0139] S3.3. Evaluate mixed-type data based on machine learning to obtain the eleventh evaluation score.

[0140] Specifically, the process of executing step S3.1 includes: building a Bayesian network structure model, determining the correspondence between the honeypot characteristics and the model node status, performing parameter learning on the model, and obtaining the first probability value of the honeypot; evaluating the performance of the Bayesian network model by comparing the ROC curves and AUC values of the models generated by different machine learning algorithms; inputting feature evidence, using the Bayesian network inference algorithm to calculate the second probability value, and comparing it with the first probability value. When the second probability value is greater than or equal to the first probability value, it is evaluated as a honeypot, otherwise it is considered not to be a honeypot, and the honeypot is evaluated to obtain a ninth evaluation score.

[0141] Furthermore, in the process of obtaining the ninth evaluation score S9, the following formula is used for calculation:

[0142] S9=W9 11 ×F91+W92×F92+w93×F93+W94×F94;

[0143] Among them, the honeypot feature integrity score The number of key features in the honeypot device data obtained is counted as n, and the total number of key features is N. The honeypot first probability value score F92 = P1 × 100, the honeypot first probability value is P1. The difference score between the second probability value and the first probability value The second probability value is P2, D = P2 - P1. When D < 0, D = 0, and the score is treated as non-negative. The model performance evaluation score F94 = A × 100, and the area under the ROC curve AUC value is A. Where W91, W92, W93, and W94 are the weights corresponding to each score. The default weights are: W91, W92 = 0.3, W93, W94 = 0.2.

[0144] Specifically, the process of executing step S3.2 includes: identifying honeypots and industrial control devices based on strong feature data, using the common features and recognition results corresponding to the identified honeypots and industrial control devices as training sets, inputting them into the deep learning model for training, and building a honeypot recognition model;

[0145] Furthermore, the online feature recognition module extracts the first feature data from the original data of different types of industrial control equipment, inputs it into the honeypot recognition model to obtain the first recognition result of the probability value. If the recognition is wrong, the erroneous result is corrected based on the model update module, and the corrected recognition result is used as a training set to input the offline honeypot recognition model for training. After the training is completed, it is reused for online recognition to obtain the tenth evaluation score.

[0146] Specifically, in the process of calculating the tenth evaluation score S10, the following formula is used:

[0147] S10=W101×F101+W102×F102+W103×F103+W104×F104;

[0148] Among them, W101, W102, W103, and W104 are the weights of the strong feature, defense feature, intrinsic feature, and network feature scores, respectively; F101, F102, F103, and F104 are the strong feature, defense feature, intrinsic feature, and network feature scores, respectively.

[0149] Furthermore, the default feature weights are set as: W101 = 0.4, W102, W103, W104 = 0.2.

[0150] Specifically, in the process of executing step S3.3, it includes: calculating the probability that the target IP is an industrial control honeypot in each step, assigning a weight to each probability based on a machine learning method; calculating the comprehensive probability of the target IP by weighted summation, evaluating the honeypot based on the comprehensive probability, and obtaining the eleventh evaluation score.

[0151] In some embodiments, the process of executing step S4 includes: accumulating the first evaluation score to the eleventh evaluation score and taking the average to obtain a comprehensive honeypot simulation quality score, where the full score of each method is 100 points. The honeypot simulation quality score simulates the behavior of a real PLC. The higher the score, the higher the simulation quality.

[0152] Furthermore, experimental verification can be used to adjust the weight of each sub-item and the proportion of different evaluation methods in the total score to adapt to the needs of different application scenarios.

[0153] While the embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations of these embodiments are possible. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the claims. Furthermore, the invention described herein is susceptible to other embodiments and may be practiced or implemented in a variety of ways.

Claims

1. A method for evaluating the simulation capability of an industrial control honeypot, characterized in that: The following steps are involved: The first evaluation score is obtained by evaluating the device system level based on the BERT model. The second evaluation score is obtained based on the log entry integrity, time series consistency, event and operation correlation, and log feature statistical calculation in the honeypot log. The third evaluation score is obtained by evaluating the device system level based on interactive verification. The fourth evaluation score is obtained by evaluating the device system level based on memory fingerprint. Based on the data packet fragmentation evaluation method to simulate normal industrial control protocol requests, the honeypot was evaluated based on the four dimensions of the general method, Modbus protocol, S7comm protocol, and IEC104 protocol, and obtained the fifth evaluation score. The industrial protocol interaction was evaluated based on the feature extraction of fuzz testing and obtained the sixth evaluation score. The industrial protocol interaction was evaluated based on the IEC104 protocol and obtained the seventh evaluation score. The industrial protocol interaction was evaluated based on the successive probability discrimination algorithm and obtained the eighth evaluation score. The evaluation of mixed-type data based on Bayesian networks received the ninth evaluation score, the evaluation of mixed-type data based on deep learning integration received the tenth evaluation score, and the evaluation of mixed-type data based on machine learning received the eleventh evaluation score. The first to eleventh evaluation scores are accumulated and averaged to obtain a comprehensive honeypot simulation quality score.

2. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, characterized in that: The process of evaluating the device system level based on the BERT model to obtain the first evaluation score includes: Acquire industrial control equipment data based on network scanning and search engines, and pre-process and annotate the industrial control equipment data; The BERT model is trained based on the labeled data to enable it to identify the characteristic entities of the industrial control honeypot. The trained BERT model is evaluated and optimized using the precision, recall, and F1 score indicators. The honeypot is evaluated based on the optimized BERT model to obtain the first evaluation score. The mathematical expression for the BERT model's evaluation of the honeypot is: S=100×(w p ·P+w r ·R+w f ·F1); Among them, S represents the BERT evaluation score, P represents the precision score, R represents the recall score, F1 represents the harmonic mean of precision and recall, and w p 、w r 、w f They represent the weights of precision, recall and comprehensive index F1 respectively.

3. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, characterized in that: The process of evaluating the device system level based on interactive verification to obtain the third evaluation score includes: Filter trusted IP nodes based on credit information, exclude known proxy nodes, CDN nodes, and egress gateway nodes, scan the remaining IP nodes, match the features in the open source honeypot feature library, and filter out the IP of the target honeypot; Perform detailed scans of target IP nodes, collect asset information, analyze the number of open ports and periodic changes in asset fingerprints, remotely log in and execute system commands, and obtain login success rates and command feedback results; The third evaluation score is obtained by scoring the IP node based on the number of open ports, login success rate, and command feedback results.

4. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, wherein: The process of evaluating the device system level based on the memory fingerprint to obtain the fourth evaluation score includes: The SSM evaluation score is obtained based on the content structure similarity and semantic conflict of the built-in operating system, the CRM evaluation score is obtained based on the memory processing service and logic execution of the memory area, the IOM evaluation score is obtained based on sensor noise and industrial process dynamics, and the fourth evaluation score is calculated based on the SSM evaluation score, CRM evaluation score and IOM evaluation score.

5. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, wherein: The process of evaluating industrial protocol interactions based on fuzz testing feature extraction to obtain the sixth evaluation score includes: The effectiveness score is obtained based on the average number of response messages and the average response number threshold. The diversity score is obtained based on the number of different types of response messages received and the response type number threshold. The balance score is obtained based on the distribution variance of the number of different types of response messages and the variance threshold. Weighting the effectiveness score, diversity score, and balance score to obtain a set of probe messages from the top m groups, filtering out a union of response message types corresponding to the probe messages from the set, and obtaining a coverage score based on the union and the number of all possible response message types; The effectiveness score, diversity score, balance score, and coverage score are added together to obtain the sixth assessment score.

6. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, wherein: The process of obtaining the seventh evaluation score by evaluating industrial protocol interaction based on the IEC104 protocol includes: Based on the monitoring tool, IEC104 protocol data packets are captured from the industrial control system, the data packets are analyzed and the message structure and interaction logic are extracted. Based on the message format of the IEC104 protocol, a malformed detection data packet is constructed. The malformed detection data packet is used to test known devices and honeypots, and the responses are recorded. An abnormal feature library and a detection data packet library are constructed, and a detection is initiated on the target device. The response data packets are collected, and based on the degree of matching between the response data packet and the abnormal feature library, the honeypot is evaluated by comparing with the preset threshold to obtain the seventh evaluation score.

7. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, wherein: The process of evaluating the industrial protocol interaction based on the successive probability discrimination algorithm to obtain the eighth evaluation score includes: Gradually introduce features, calculate the conditional probability that the device is a honeypot under each feature, and compare it with the set threshold to decide whether to identify the device as a honeypot; The conditional probability is calculated using the first feature. If it is greater than the set threshold, the device is considered a honeypot. If it is less than the threshold, the next feature is introduced and the conditional probability is calculated until all features are used or the conditional probability exceeds the threshold. For devices running the protocol, the ratio of the conditional probability to the total number of samples is calculated to determine whether the device is a honeypot and obtain the eighth evaluation score.

8. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, wherein: The process of evaluating mixed-type data based on the Bayesian network to obtain the ninth evaluation score includes: Build a Bayesian network structure model, determine the correspondence between honeypot features and model node states, learn model parameters, and obtain the first probability value of the honeypot; Evaluate the performance of the Bayesian network model by comparing the ROC curves and AUC values of models generated by different machine learning algorithms; Input feature evidence, use the Bayesian network inference algorithm to calculate the second probability value, and compare it with the first probability value. When the second probability value is greater than or equal to the first probability value, it is considered to be a honeypot, otherwise it is considered not to be a honeypot. The honeypot is evaluated to obtain the ninth evaluation score.

9. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, wherein: The process of evaluating mixed-type data based on deep learning integration to obtain the tenth evaluation score includes: Identify honeypots and industrial control devices based on strong feature data. Use the common features and recognition results corresponding to the identified honeypots and industrial control devices as training sets, input them into the deep learning model for training, and build a honeypot recognition model. The online feature recognition module extracts the first feature data from the original data of different types of industrial control equipment, inputs it into the honeypot recognition model to obtain the first recognition result with a probability value. If the recognition is wrong, the wrong result is corrected based on the model update module, and the corrected recognition result is used as a training set to input the offline honeypot recognition model for training. After the training is completed, it is reused for online recognition to obtain the tenth evaluation score.

10. The method for evaluating the simulation capability of an industrial control honeypot according to claim 1, wherein: The eleventh evaluation score was obtained by evaluating mixed-type data based on machine learning, including: Calculate the probability that the target IP is an industrial control honeypot in each step, and assign a weight to each probability based on a machine learning method; calculate the comprehensive probability of the target IP by weighted summation, evaluate the honeypot based on the comprehensive probability, and obtain an eleventh evaluation score.

Images