Automatic industrial control protocol reverse engineering system and method based on global voting expert algorithm

Abstract

The present invention discloses an automatic industrial control protocol reverse engineering system and method based on a global voting expert algorithm, which relates to the field of network protocol technology. The system divides a data packet into inferred fields of fixed length and variable length fields by accurately identifying the message boundaries of the industrial control protocol, judges the confusion between the fields by analyzing the logical relationship of the inferred fields, classifies similar messages into clusters according to the field structure and the distribution of function codes, and evaluates the diversity of function codes. The accuracy of the analysis is evaluated according to the logical confusion and the distribution of function codes, and is divided into high, medium and low levels. For medium-accuracy analysis, an abnormality prediction module predicts the abnormality of the accuracy value and decides whether to adjust the algorithm parameters or perform local optimization. This not only improves the accuracy and stability of the protocol analysis, but also solves the shortcomings of existing tools in processing protocol diversity and clustering errors, and ensures that the generated protocol format tree can accurately reflect the structure of complex industrial protocols.

Description

Technical Field

[0001] The present invention relates to the technical field of network protocols, and in particular to an automatic industrial control protocol reverse engineering system and method based on a global voting expert algorithm. Background Art

[0002] Industrial control systems are an important part of many critical infrastructures, and they also have very stringent security requirements. With the rapid development of industrial control systems, the use of computer networks by industrial control systems has increased significantly, bringing many security issues. Protocol security is one of the most important security issues. Many industrial protocols are unknown, which hinders the firewall from parsing and analyzing network traffic, thus bringing huge challenges to intrusion detection, deep packet inspection and traffic management. One way to solve this problem is reverse engineering technology. However, previous work is mainly used for traditional network protocols and is not very suitable for reverse industrial protocols. In the prior art, an unsupervised protocol analysis tool that automatically reverses the protocol format from network traffic is proposed, and an extended global voting expert algorithm is applied to infer the boundaries of each field in the industrial control protocol. The types of these fields are derived by statistical methods. Then, it classifies the messages into subclusters according to their field types and infers the format of each subcluster. Finally, the tool merges all the results to obtain the format tree of the protocol. The tool's reverse industrial protocol is mainly divided into three stages: segmentation, classification, and summary. The tool first segments the original data packet into multiple parts and infers the fields of the protocol. The properties (offset, length, etc.) and meaning (length, sequence number, etc.) of these two fields. Then, it groups similar formats into a cluster, and eventually makes each cluster converge to one format. Finally, the tool combines all the formats to get the protocol format tree.

[0003] The prior art has the following deficiencies:

[0004] In the process of message clustering, the classification algorithms of existing tools may not be able to handle the diversity of protocols. Since many industrial protocol message formats are similar but the functions are different, it is difficult for existing technologies to effectively distinguish these subtle differences, resulting in messages with different functions being mistakenly grouped into one cluster, or messages with the same function being incorrectly grouped. At the same time, clustering errors may cause the generated protocol format tree to be inaccurate. If messages with different functions are grouped into the same cluster, the field definitions in the format tree will not correctly reflect the actual protocol structure, especially the roles of key fields such as function codes and serial numbers will be confused. Ultimately, the generated format tree will lack a clear definition of the internal logic and hierarchy of the protocol, affecting the complete parsing of the protocol. This will seriously hinder the subsequent understanding and implementation of the protocol, especially when it comes to complex industrial protocols, the entire reverse process may be ineffective. Summary of the invention

[0005] The purpose of the present invention is to provide an automatic industrial control protocol reverse system based on a global voting expert algorithm to solve the shortcomings of the background technology.

[0006] On the other hand, the present invention also provides an automatic industrial control protocol reverse method based on a global voting expert algorithm.

[0007] In order to achieve the above object, the present invention provides the following technical solution: an automatic industrial control protocol reverse system based on a global voting expert algorithm, comprising:

[0008] A field boundary identification module is used to obtain the original data packet of the industrial control protocol from the network, and use the global voting expert algorithm to perform message boundary identification on the original data packet, wherein the message boundary identification includes inferring the field boundary in the data packet through a sliding window and a statistical analysis method, and dividing the data packet into inferred fields according to the inference result, wherein the inferred fields include fixed-length fields and variable-length fields;

[0009] The logic evaluation module is used to determine the degree of semantic anomaly between adjacent fields by statistically analyzing the attributes and meanings of each inferred field, and to evaluate the degree of confusion of the logical relationship between adjacent fields based on the degree of semantic anomaly;

[0010] The message clustering evaluation module is used to classify the inferred fields with similar field structures into a cluster and evaluate the diversity of function codes in the same cluster by checking the distribution of function code fields within the cluster;

[0011] A parsing accuracy evaluation module is used to evaluate the accuracy of protocol parsing according to the confusion degree of logical relations between adjacent fields and the diversity of function codes in the same cluster, and to divide the accuracy of protocol parsing into high-accuracy parsing, medium-accuracy parsing and low-accuracy parsing according to the evaluation results;

[0012] Adjust the processing module for high-accuracy parsing, which is directly used to generate the final protocol format tree;

[0013] The adjustment processing module is also used to perform several iterative calculations for low-accuracy analysis through a global voting expert algorithm and re-derive field boundaries using context information;

[0014] The automatic industrial control protocol reverse system also includes: an abnormality prediction module, which is used to predict and analyze the abnormality degree of protocol analysis accuracy within a fixed time period for medium accuracy analysis. If the abnormality degree is high, the parameters of the global voting expert algorithm are adjusted; if the abnormality degree is low, the cluster is locally optimized.

[0015] Preferably, in the field boundary identification module, a global voting expert algorithm is used to determine the field boundary of each message, and a sliding window method is used to move byte by byte in the message to analyze the pattern of each position. In each window, the field boundary is voted on through a global voting mechanism, and the voting is based on the byte patterns and message structure characteristics that appear frequently in the field. During the movement of the sliding window, global context information is used to analyze the global pattern in the message, and the boundary information of the local message is compared with the global context. After the voting results are accumulated, the position with the highest number of votes is identified as a candidate point for the field boundary; the boundary identified by the global voting expert algorithm is inferred to determine whether it is accurate, and the candidate field boundary is verified. If the boundary point appears repeatedly in multiple messages and the number of votes is higher than any boundary, it is the boundary of a fixed field.

[0016] Preferably, in the logic evaluation module, according to the correlation between the sequence number and the timestamp and the matching between the length field and the subsequent data field, the correlation anomaly index between adjacent fields is calculated to evaluate the confusion of the logical relationship between adjacent fields. The method for obtaining the correlation anomaly index is:

[0017] Get the length of the data segment defined in the protocol and build a time series ; and the actual data length carried in each protocol data packet, construct time series B ; Use the DTW algorithm to calculate the minimum matching cost between the two time series, determine the matching degree between the length field and the data field, and create a cost matrix D of size n×m, where D(i,j) represents the distance between the i-th element in sequence A and the j-th element in sequence B, with the starting point being D(1,1) and the end point being D(n,m); use the distance function , for each sequence position (i, j), calculate the squared difference distance between the i-th value in sequence A and the j-th value in sequence B, expressed as: ; By recursive formula, calculate the cumulative cost matrix D(i,j), which means from arrive and arrive The minimum matching cost is expressed as: ; Among them, the min function ensures that the cumulative cost of the path is minimized; the recursive formula finds the optimal path through three path directions: from the top, from the left, and from the upper left, and accumulates the current cost; the cumulative cost of the initial point is: ; The cumulative cost of the first row and column is: ; ; By calculating the final cumulative cost, the matching degree of the two sequences is determined, that is, the matching degree of adjacent fields; the final DTW distance appears in the lower right corner of the matrix D(n,m), and its value represents the total cost of the minimum matching path between sequences A and B : ;Will Normalized to the correlation anomaly index, the expression is: ; In the formula, is the correlation anomaly index.

[0018] Preferably, in the message clustering evaluation module, a dominant function code ratio variation index is generated according to the frequency of occurrence of each function code in the cluster, and the diversity of function codes in the same cluster is evaluated. The method for obtaining the dominant function code ratio variation index is:

[0019] Mark the function code distribution in the cluster as ,in, Indicates function code Frequency of occurrence in the cluster, dominant function code is the function code with the highest frequency; calculate the total entropy of the function code distribution in the cluster , the expression is: ; k is the number of different function codes in the cluster, and the dominant function code is calculated , the expression is: ; Calculate the entropy difference between the total entropy and the entropy of the dominant function code , the expression is: ; The entropy difference represents the difference between the overall diversity of the function code and the concentration of the dominant function code. The dominant function code ratio variation index is calculated, and the expression is: ; In the formula, is the dominant function code ratio variation index.

[0020] Preferably, in the parsing accuracy evaluation module, the accuracy of protocol parsing is evaluated based on the confusion degree of logical relationships between adjacent fields and the diversity of function codes in the same cluster;

[0021] The correlation anomaly index and the dominant function code ratio variation index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the accuracy value label of the protocol parsing for each group of first eigenvectors as the prediction target, and minimizes the sum of the prediction errors of the accuracy value labels of all protocol parsing as the training target. The machine learning model is trained until the sum of the prediction errors converges, and the model training is stopped. The accuracy value of the protocol parsing is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0022] Preferably, the acquired accuracy value of the protocol analysis is compared with a gradient standard threshold, the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the accuracy value of the protocol analysis is compared with the first standard threshold and the second standard threshold respectively;

[0023] If the accuracy value of the protocol analysis is greater than the second standard threshold, it is classified as a high-accuracy analysis and a high-accuracy analysis signal is generated; if the accuracy value of the protocol analysis is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is classified as a medium-accuracy analysis and a medium-accuracy analysis signal is generated; if the accuracy value of the protocol analysis is less than the first standard threshold, it is classified as a low-accuracy analysis and a low-accuracy analysis signal is generated.

[0024] Preferably, in the abnormal prediction module, for medium accuracy analysis, the abnormal degree of protocol analysis accuracy within a fixed time period is predicted and analyzed;

[0025] When the accuracy value of the protocol analysis generated within a fixed time period is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is classified as medium accuracy analysis, and the accuracy values ​​of the protocol analysis generated in subsequent fixed time periods are collected, and a data set is established. The mean and standard deviation of the data set are calculated, and after analysis, the abnormal degree of the protocol analysis accuracy in subsequent fixed time periods is predicted.

[0026] Preferably, if the mean of the accuracy values ​​in the data set is greater than or equal to the reference threshold of the mean of the accuracy values, and the standard deviation of the accuracy values ​​is less than the reference threshold of the standard deviation of the accuracy values, it means that the analysis result is stable and the accuracy of the analysis is high. At this time, no warning signal is generated, the abnormality is low, and the cluster is locally optimized;

[0027] If the mean of the accuracy values ​​is greater than or equal to the reference threshold of the mean of the accuracy values, and the standard deviation of the accuracy values ​​is greater than or equal to the reference threshold of the standard deviation of the accuracy values, it indicates that there is a local parsing error, and a warning signal is generated at this time, the abnormality is high, and the parameters of the global voting expert algorithm are adjusted;

[0028] If the mean of the accuracy values ​​is less than the reference threshold of the mean of the accuracy values, and the standard deviation of the accuracy values ​​is greater than or equal to the reference threshold of the standard deviation of the accuracy values, the accuracy value of the protocol analysis is low and the distribution is unstable. At this time, a warning signal is generated, the abnormality is high, and comprehensive adjustments are made;

[0029] If the mean accuracy value is less than the reference threshold of the mean accuracy value, and the standard deviation of the accuracy value is less than the reference threshold of the standard deviation of the accuracy value, the accuracy value of the protocol parsing is low but the distribution is stable. At this time, no warning signal is generated and the abnormality is low; local optimization is performed.

[0030] On the other hand, the present invention also provides an automatic industrial control protocol reverse engineering method based on a global voting expert algorithm, comprising:

[0031] Obtaining an original data packet of an industrial control protocol from a network, and using a global voting expert algorithm to perform message boundary identification on the original data packet, wherein the message boundary identification includes inferring field boundaries in the data packet through a sliding window and statistical analysis, and dividing the data packet into inferred fields according to the inferred result, wherein the inferred fields include fixed-length fields and variable-length fields;

[0032] By statistically analyzing the attributes and meanings of each inferred field, the degree of semantic anomaly between adjacent fields is determined, and the degree of confusion of the logical relationship between adjacent fields is evaluated based on the degree of semantic anomaly;

[0033] For the segmented messages, messages with similar field structures are classified into a cluster, and the diversity of function codes in the same cluster is evaluated by examining the distribution of function code fields within the cluster;

[0034] According to the confusion degree of logical relationship between adjacent fields and the diversity of function codes in the same cluster, the accuracy of protocol parsing is evaluated, and the accuracy of protocol parsing is divided into high-accuracy parsing, medium-accuracy parsing and low-accuracy parsing according to the evaluation results;

[0035] For high-accuracy parsing, it is directly used to generate the final protocol format tree; for low-accuracy parsing, several iterative calculations are performed through the global voting expert algorithm, and the field boundaries are re-derived using context information;

[0036] For medium accuracy parsing, the degree of abnormality of the protocol parsing accuracy within a fixed time period is predicted and analyzed. If the degree of abnormality is high, the parameters of the global voting expert algorithm are adjusted. If the degree of abnormality is low, the cluster is locally optimized.

[0037] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0038] 1. The present invention uses a sliding window and voting mechanism to accurately identify the field boundaries of the message through the field boundary identification module. The logic evaluation module ensures the accuracy of the parsing result by analyzing the semantic anomalies and logical relationship confusion between adjacent fields. The message clustering evaluation module correctly classifies similar messages through the distribution diversity evaluation of the function code to prevent messages with different functions from being confused in the same cluster. The parsing accuracy evaluation module combines the correlation anomaly index and the dominant function code ratio variation index to accurately predict the accuracy of the parsing based on the machine learning model, and further optimizes the low-accuracy parsing by adjusting the processing module.

[0039] 2. The present invention provides a dynamic analysis mechanism for medium accuracy analysis through an abnormal prediction module. By monitoring the accuracy value of the analysis within a fixed time period, the abnormality degree is predicted and it is decided whether to adjust the global voting expert algorithm parameters or perform cluster local optimization. The overall solution not only improves the accuracy and stability of protocol analysis, but also the generated protocol format tree can accurately reflect the structure of complex industrial protocols, ensuring that the protocol analysis results provide a reliable basis for subsequent implementation and understanding. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] Figure 1 is a flow chart of the method of the present invention;

[0042] Figure 2 It is a system module diagram of the present invention;

[0043] Figure 3 This is a diagram to illustrate relocation. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] For examples, see Figure 1 , Figure 2 and Figure 3 As shown, the automatic industrial control protocol reverse engineering system based on the global voting expert algorithm described in this embodiment includes a field boundary recognition module, a logic evaluation module, a message clustering evaluation module, a parsing accuracy evaluation module, an adjustment processing module and an abnormality prediction module;

[0046] A field boundary identification module is used to obtain the original data packet of the industrial control protocol from the network, and use the global voting expert algorithm to perform message boundary identification on the original data packet. The message boundary identification includes inferring the field boundary in the data packet through sliding windows and statistical analysis, and dividing the data packet into inferred fields according to the inference results. The inferred fields include fixed-length fields and variable-length fields.

[0047] The logic evaluation module is used to determine the degree of semantic anomaly between adjacent fields by statistically analyzing the attributes and meanings of each inferred field, and to evaluate the degree of confusion of the logical relationship between adjacent fields based on the degree of semantic anomaly;

[0048] The message clustering evaluation module is used to classify the inferred fields with similar field structures into a cluster and evaluate the diversity of function codes in the same cluster by checking the distribution of function code fields within the cluster;

[0049] A parsing accuracy evaluation module is used to evaluate the accuracy of protocol parsing according to the confusion degree of logical relations between adjacent fields and the diversity of function codes in the same cluster, and to divide the accuracy of protocol parsing into high-accuracy parsing, medium-accuracy parsing and low-accuracy parsing according to the evaluation results;

[0050] The adjustment processing module is used to directly generate the final protocol format tree for high-accuracy parsing; for low-accuracy parsing, the global voting expert algorithm is used to perform several iterative calculations and re-derive the field boundaries using context information;

[0051] The anomaly prediction module is used to predict and analyze the degree of anomaly of the protocol parsing accuracy within a fixed time period for medium accuracy parsing. If the degree of anomaly is high, the parameters of the global voting expert algorithm are adjusted. If the degree of anomaly is low, the cluster is locally optimized.

[0052] Among them, in the field boundary identification module, the original data packet of the industrial control protocol is obtained from the network, and the message boundary is identified using the global voting expert algorithm. The field boundaries in the data packet are inferred through sliding windows and statistical analysis methods, and the data packet is divided into fixed-length fields and variable-length fields according to the inference results.

[0053] Use data capture tools (such as Wireshark or custom network monitoring tools) to capture the data packet flow in the industrial control network, identify specific industrial control protocols (such as Modbus, IEC104, Ethernet / IP) and filter related traffic to ensure that only data packets related to the target protocol are obtained. Store the captured data packets in binary format files for subsequent processing.

[0054] Preprocess the captured raw data packets to prepare for parsing. Clean up the redundant information in the data packets and remove the data or interference packets that are not related to the protocol. Standardize the data packets and convert them into a format suitable for processing by the global voting expert algorithm, such as reading data in the form of byte streams. Perform preliminary marking on the data packets to identify possible message boundaries (such as header information, specific protocol identifiers, etc.).

[0055] The global voting expert algorithm (GVE algorithm) is used to determine the field boundaries of each message. The sliding window method is used to move byte by byte in the message and analyze the pattern of each position. In each window, the field boundaries are voted for through the global voting mechanism. The voting is based on the byte patterns that appear frequently in the field, the message structure characteristics, etc. During the sliding window movement, the global context information is used to analyze the global pattern in the message, and the boundary information of the local message is compared with the global context to enhance the accuracy of the boundary. After the voting results are accumulated, the position with the highest number of votes is identified as the candidate point of the field boundary. The boundaries identified by the GVE algorithm are inferred to determine whether they are accurate. The candidate field boundaries are verified and the stability of the field is analyzed to ensure its consistency in different data packets. If some boundary points appear repeatedly in multiple messages and the number of votes is significantly higher than any boundary, the boundary point is considered to be the boundary of a fixed field (such as the function code and serial number field). The context information is used to confirm the boundary of a variable field. For example, the data content field may have a variable length, and its boundary needs to be further confirmed by detecting the change in the data length.

[0056] Based on the inferred boundaries, the message is split into fixed-length and variable-length fields. Based on the boundary inference results of the GVE algorithm, the fixed-length fields are identified. These fields are usually protocol header information, such as length fields, function code fields, and sequence number fields, which have consistent lengths. Identify variable-length fields, such as data content fields, whose lengths may vary with different message contents. Determine the length of variable-length fields by analyzing the relationship between the length field and the actual data size in the message. Classify and split all fields in the message into fixed-length and variable-length fields, and record the position and length of each field. Mark the split fields to facilitate subsequent message parsing and clustering analysis.

[0057] Save the segmented messages and field data to provide a basis for subsequent protocol parsing and clustering. Store the segmented message structure in a standard format, including information such as the boundaries, type (fixed or variable), and length of each field. Add tags to each message to indicate its message type, function code, and other related attributes. Prepare for the subsequent clustering step, based on these segmented fields, to perform function code field analysis and message classification.

[0058] The logic evaluation module statistically analyzes the attributes and meanings of each inferred field, determines the degree of semantic anomaly between adjacent fields, and evaluates the confusion of the logical relationship between adjacent fields.

[0059] Each inferred field is classified into categories, such as function code field, length field, sequence number field, or data field. Based on the data packets of multiple messages, the characteristic attributes of each field are analyzed, including the length of the field, the value range, the frequency of change, etc. Fixed fields (such as function codes and flags) usually have consistent values ​​or patterns; variable fields (such as data fields) are characterized by changes in length and content.

[0060] In the protocol, the value or type of some fields depends on the previous and next fields. For example, the length field should match the actual length of the data field that follows it, and the function code field should be consistent with the operation type represented by the subsequent field. Check the rationality of the field value through the context. By analyzing the combination of adjacent fields, verify whether their logical relationship conforms to the general semantics of the protocol. For example, the type and order of the fields after the function code are usually fixed. If the order is abnormal, it may indicate that the boundary identification is incorrect. Use the normal field combination model in the context to determine whether there is an abnormal field combination. If the value or type of an adjacent field does not conform to the logic of the common protocol, such as a large jump in the serial number field, it indicates a semantic abnormality.

[0061] According to the correlation between the sequence number and the timestamp and the matching between the length field and the subsequent data field, the correlation anomaly index between adjacent fields is calculated to evaluate the confusion of the logical relationship between adjacent fields. The correlation anomaly index is obtained as follows:

[0062] Get the length of the data segment defined in the protocol and construct the time series A (length field sequence), recorded as ; and the actual data length carried in each protocol data packet, construct time series B (sequence of subsequent data field lengths), recorded as ; Use the DTW algorithm to calculate the minimum matching cost between the two time series, determine the matching degree between the length field and the data field, and create a cost matrix D of size n×m, where D(i,j) represents the distance between the i-th element in sequence A and the j-th element in sequence B, with the starting point being D(1,1) and the end point being D(n,m); use a distance function , usually the Euclidean distance, for each sequence position (i, j), calculate the square difference distance between the i-th value in sequence A and the j-th value in sequence B, the expression is: ; By recursive formula, calculate the cumulative cost matrix D(i,j), which means from arrive and arrive The minimum matching cost is expressed as: ; The min function ensures that the cumulative cost of the path is minimized. This recursive formula finds the optimal path through three possible path directions: from the top, from the left, and from the upper left, and accumulates the current cost. The cumulative cost of the initial point is: ; The cumulative cost of the first row and column is: ; ; By calculating the final cumulative cost, the matching degree of the two sequences is determined, that is, the matching degree of adjacent fields (length field and data field). The larger the cumulative cost, the worse the matching degree of the two fields and the higher the abnormal correlation. The final DTW distance appears in the lower right corner of the matrix D(n,m), and its value represents the total cost of the minimum matching path between sequences A and B. : ; A large value indicates that the length field does not match the length of the subsequent data field, indicating an abnormal correlation. Normalized to the correlation anomaly index, the expression is: ; In the formula, is the correlation anomaly index.

[0063] The obtained correlation anomaly index is compared with the preset correlation anomaly index reference threshold. If the correlation anomaly index is greater than or equal to the preset correlation anomaly index reference threshold, it means that there is confusion in the logical relationship between the fields and further optimization and adjustment are required; if the correlation anomaly index is less than the preset correlation anomaly index reference threshold, it means that there is no confusion in the logical relationship between the fields and no additional processing is required.

[0064] The message clustering evaluation module is used to classify the inferred fields with similar field structures into a cluster after the segmentation, and evaluate the diversity of function codes in the same cluster by checking the distribution of function code fields in the cluster.

[0065] During the protocol parsing process, the segmented messages need to be clustered based on similar field structures. The function code field is an important field in the protocol message used to identify the function or operation type. By checking the distribution of the function code field within the cluster, the diversity of function codes in the same cluster can be evaluated to ensure the accuracy of the message cluster.

[0066] Extract the field structure of each message from the segmented messages. This includes the length, position, and type of the field (such as function code, serial number, data field, etc.). Identify the function code field in the message. The function code field is usually a flag bit in the protocol message, indicating the specific function of the message (such as read, write, control, etc.). Classify messages with similar field structures into the same cluster. Calculate the similarity between each message based on the similarity of the field structure (such as the type, length, order of the field, etc.). Messages with high similarity are considered to belong to the same type. Use a clustering algorithm (such as K-means, hierarchical clustering, or DBSCAN) to cluster messages with high similarity and group messages with similar field structures into the same cluster. Within each cluster, check the distribution of the function code field and evaluate the diversity of the function code. In each cluster, count the function code field values ​​in all messages. Record the frequency of occurrence of each function code in the cluster.

[0067] The dominant function code ratio variation index is generated according to the frequency of occurrence of each function code in the cluster to evaluate the diversity of function codes in the same cluster. The method for obtaining the dominant function code ratio variation index is as follows:

[0068] Mark the function code distribution in the cluster as ,in, Indicates function code Frequency of occurrence in the cluster, dominant function code is the function code with the highest frequency; calculate the total entropy of the function code distribution in the cluster , the expression is: ; k is the number of different function codes in the cluster, and the dominant function code is calculated , the expression is: ; Calculate the entropy difference between the total entropy and the entropy of the dominant function code , the expression is: ; The entropy difference represents the difference between the overall diversity of the function code and the concentration of the dominant function code. The larger the entropy difference, the more significant the impact of the dominant function code on the cluster and the more uneven the distribution of the function code; the smaller the entropy difference, the relatively uniform distribution of the function code; calculate the dominant function code ratio variation index, the expression is: ; In the formula, is the dominant function code ratio variation index.

[0069] The larger the dominant function code ratio variation index is, the lower the function code diversity is, the dominant function code has a stronger influence on the cluster, and other function codes appear less frequently. The smaller the dominant function code ratio variation index is, the higher the function code diversity is, the function codes in the cluster are evenly distributed, and multiple function codes participate in the function division in the cluster.

[0070] Parsing accuracy evaluation module: evaluates the accuracy of protocol parsing based on the confusion of logical relationships between adjacent fields and the diversity of function codes in the same cluster, and divides the accuracy of protocol parsing into high-accuracy parsing, medium-accuracy parsing, and low-accuracy parsing based on the evaluation results.

[0071] The correlation anomaly index and the dominant function code ratio variation index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the accuracy value label of the protocol parsing for each group of first eigenvectors as the prediction target, and minimizes the sum of the prediction errors of the accuracy value labels of all protocol parsing as the training target. The machine learning model is trained until the sum of the prediction errors converges, and the model training is stopped. The accuracy value of the protocol parsing is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0072] The method for obtaining the accuracy value of the protocol analysis is: from the first feature vector training data of the trained machine learning model, obtain the corresponding function expression: ; In the formula, is the output function of the model, is the correlation anomaly index, is the dominant function code ratio variation index, For the accuracy of protocol parsing.

[0073] The acquired accuracy value of the protocol analysis is compared with a gradient standard threshold, wherein the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the accuracy value of the protocol analysis is compared with the first standard threshold and the second standard threshold respectively;

[0074] If the accuracy value of the protocol analysis is greater than the second standard threshold, it is classified as a high-accuracy analysis and a high-accuracy analysis signal is generated; if the accuracy value of the protocol analysis is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is classified as a medium-accuracy analysis and a medium-accuracy analysis signal is generated; if the accuracy value of the protocol analysis is less than the first standard threshold, it is classified as a low-accuracy analysis and a low-accuracy analysis signal is generated.

[0075] Adjust the processing module: For high-accuracy parsing, it is directly used to generate the final protocol format tree; for low-accuracy parsing, several iterative calculations are performed through the global voting expert algorithm, and the field boundaries are re-derived using context information.

[0076] When the accuracy of protocol parsing is classified as high accuracy, it means that the current field boundary inference, function code distribution, etc. are very reliable and the parsing results are credible. Therefore, these high-accuracy parsing can be directly used to generate the final protocol format tree.

[0077] The messages in high-accuracy parsing have been parsed to obtain accurate field structures and function code distributions, and these messages will be classified into corresponding function clusters. The protocol format tree is a hierarchical structure that represents the hierarchical relationship of different fields in the protocol. Each node represents a protocol field, including field type, length, order, and other information. Starting from the most basic message header, each field (such as function code, sequence number, data field) is inserted into the protocol format tree in the order of parsing. When all fields and messages are processed, the format tree represents the structure of the entire protocol, including field hierarchy, length, type, etc.

[0078] Due to the high accuracy of the analysis, these analysis results are directly used to build the protocol format tree. Since the boundaries and types of all fields have been correctly derived, these high-accuracy analyses can ensure that the generated protocol format tree is very accurate and can truly reflect the structure of the protocol.

[0079] When protocol parsing is classified as low accuracy, it means that there may be major errors in the current parsing process, inaccurate field boundary inference, and abnormal function code distribution. At this time, it is necessary to use the Global Voting Expert (GVE) algorithm to iterate multiple times to re-derive field boundaries and use context information to improve parsing accuracy.

[0080] Use the global voting expert algorithm to re-perform sliding window analysis on the captured packets to infer potential field boundaries. For each byte position, use the statistical pattern of the global context to vote to find possible field boundary points. The size of the sliding window and the voting results of the pattern within the window determine the initial candidate position for boundary inference. During the iterative process of low-accuracy parsing, use information from other messages to assist in inferring the field boundaries of the current message. For example, if the function codes of some messages have been accurately parsed, these results can be used as context information to assist in inferring incorrectly parsed messages. Re-vote on the field boundaries of each message based on the function code distribution in the context, field length pattern, etc. Multiple iterations ensure that the boundary inference is more accurate. The global voting expert algorithm will adjust the field boundaries in multiple iterations. In each iteration, the voting mechanism is corrected using the boundary inference results of the previous round to gradually optimize the accuracy of the field boundaries. When the voting results tend to be stable, that is, the boundaries of all fields no longer change after multiple rounds of voting, the iteration process ends. The re-derived field boundaries will match the length and type of adjacent fields to ensure that the relationship between all fields is reasonable. Combined with the re-derived field boundaries, the distribution of function codes is analyzed to ensure that the message type corresponding to each function code is reasonable. Through the parsing results after iterative calculation, the accuracy of the parsing can be evaluated again and reclassified as medium or high accuracy. If the parsing accuracy is greatly improved, these messages can be used to generate the protocol format tree.

[0081] The global voting expert algorithm gradually corrects the derivation of field boundaries through multiple iterations to ensure more accurate parsing results. The introduction of contextual information can improve the relevance between messages, help correct errors in the parsing process, and improve the quality of low-accuracy parsing.

[0082] Anomaly prediction module: For medium accuracy analysis, the abnormal degree of protocol analysis accuracy within a fixed time period is predicted and analyzed. If the abnormal degree is high, the parameters of the global voting expert algorithm are adjusted. If the abnormal degree is low, the cluster is locally optimized.

[0083] When the accuracy value of the protocol analysis generated within a fixed time period is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is classified as medium accuracy analysis, and the accuracy values ​​of the protocol analysis generated in subsequent fixed time periods are collected, and a data set is established. The mean and standard deviation of the data set are calculated, and after analysis, the abnormal degree of the protocol analysis accuracy in subsequent fixed time periods is predicted.

[0084] If the mean accuracy value in the data set is greater than or equal to the reference threshold of the mean accuracy value, and the standard deviation of the accuracy value is less than the reference threshold of the standard deviation of the accuracy value, the accuracy value of the protocol parsing is high and the distribution is relatively concentrated, indicating that the parsing result is relatively stable and the accuracy of the parsing is high. At this time, no warning signal is generated. In this case, the abnormality of the protocol parsing is low. The current cluster can be locally optimized, such as fine-tuning the function code distribution and field boundaries to improve the overall quality of the parsing;

[0085] If the mean of the accuracy values ​​is greater than or equal to the reference threshold of the mean of the accuracy values, and the standard deviation of the accuracy values ​​is greater than or equal to the reference threshold of the standard deviation of the accuracy values, the accuracy value of the protocol parsing is high, but the distribution is unstable and the parsing results fluctuate greatly, indicating that there may be local parsing errors. At this time, an early warning signal is generated. In this case, the abnormality is high, indicating that the inconsistency of the parsing results has increased. At this time, it is necessary to adjust the parameters of the global voting expert algorithm, such as adjusting the size of the sliding window or the voting weight, to stabilize the field boundary inference and reduce fluctuations;

[0086] If the mean accuracy value is less than the reference threshold of the mean accuracy value, and the standard deviation of the accuracy value is greater than or equal to the reference threshold of the standard deviation of the accuracy value, the accuracy value of the protocol analysis is low and the distribution is unstable, the analysis quality is poor and the fluctuation is large, and there may be serious analysis errors. At this time, an early warning signal is generated. In this case, the abnormality is high, indicating that the quality of the analysis result is unreliable and needs to be fully adjusted. The key parameters of the global voting expert algorithm, such as window size, function code weight, etc., should be adjusted, and the message mode should be re-evaluated, and multiple iterative calculations should be performed to correct the analysis errors;

[0087] If the mean accuracy value is less than the reference threshold of the mean accuracy value, and the standard deviation of the accuracy value is less than the reference threshold of the standard deviation of the accuracy value, the accuracy value of the protocol parsing is low, but the distribution is stable, indicating that although the parsing result is inaccurate, the result consistency is good, and there may be systematic errors. At this time, no warning signal is generated. In this case, the abnormality is low; local optimization is performed to focus on processing some obvious errors in the parsing (such as incorrect classification of function codes or systematic deviations of field boundaries). There is no need to adjust the entire system, only a small range of specific cluster parsing strategies need to be optimized.

[0088] In this embodiment, the original data packet of the industrial control protocol is first obtained from the network through the field boundary identification module, the message boundary is identified by the global voting expert algorithm and the sliding window technology, and the data packet is divided into fixed-length and variable-length fields. Then, the inferred field attributes are analyzed by the logic evaluation module to determine the semantic anomalies and logical confusion of adjacent fields. The message clustering evaluation module classifies messages with similar field structures into clusters to evaluate the diversity of function code distribution. The parsing accuracy evaluation module divides the accuracy of protocol parsing into three levels: high, medium, and low according to the logical confusion and function code diversity. For high-accuracy parsing, it is directly used to generate the protocol format tree; for low-accuracy parsing, the field boundary is derived through multiple iterations of the global voting expert algorithm. The abnormal prediction module of the medium-accuracy parsing analyzes the degree of abnormality according to the accuracy value within the time period. If the degree of abnormality is high, the algorithm parameters are adjusted, and if the degree of abnormality is low, local optimization is performed to improve the parsing quality.

[0089] In another embodiment, the present application provides an automatic industrial control protocol reverse engineering method based on a global voting expert algorithm, comprising:

[0090] Obtaining an original data packet of an industrial control protocol from a network, and using a global voting expert algorithm to identify message boundaries of the original data packet, including inferring field boundaries in the data packet by a sliding window and a statistical analysis method, and dividing the data packet into inferred fields according to the inference result, wherein the inferred fields include a fixed-length field and a variable-length field;

[0091] By statistically analyzing the attributes and meanings of each inferred field, the degree of semantic anomaly between adjacent fields is determined, and the degree of confusion of the logical relationship between adjacent fields is evaluated based on the degree of semantic anomaly;

[0092] For the segmented inferred fields, the inferred fields with similar field structures are classified into a cluster, and the diversity of function codes in the same cluster is evaluated by examining the distribution of function code fields within the cluster;

[0093] According to the confusion degree of logical relationship between adjacent fields and the diversity of function codes in the same cluster, the accuracy of protocol parsing is evaluated, and the accuracy of protocol parsing is divided into high-accuracy parsing, medium-accuracy parsing and low-accuracy parsing according to the evaluation results;

[0094] For high-accuracy parsing, it is directly used to generate the final protocol format tree; for low-accuracy parsing, several iterative calculations are performed through the global voting expert algorithm, and the field boundaries are re-derived using context information;

[0095] For medium accuracy parsing, the degree of abnormality of the protocol parsing accuracy within a fixed time period is predicted and analyzed. If the degree of abnormality is high, the parameters of the global voting expert algorithm are adjusted. If the degree of abnormality is low, the cluster is locally optimized.

[0096] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0097] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0098] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0099] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0102] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0103] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An automatic industrial control protocol reverse system based on a global voting expert algorithm, characterized by: include: A field boundary identification module is used to obtain the original data packet of the industrial control protocol from the network, and use the global voting expert algorithm to perform message boundary identification on the original data packet, wherein the message boundary identification includes inferring the field boundary in the data packet through a sliding window and statistical analysis, and dividing the data packet into inferred fields according to the inference result, wherein the inferred fields include fixed-length fields and variable-length fields; The logic evaluation module is used to determine the degree of semantic anomaly between adjacent fields by statistically analyzing the attributes and meanings of each inferred field, and to evaluate the degree of confusion of the logical relationship between adjacent fields based on the degree of semantic anomaly; The message clustering evaluation module is used to classify the inferred fields with similar field structures into a cluster and evaluate the diversity of function codes in the same cluster by checking the distribution of function code fields within the cluster; A parsing accuracy evaluation module is used to evaluate the accuracy of protocol parsing according to the confusion degree of logical relations between adjacent fields and the diversity of function codes in the same cluster, and to divide the accuracy of protocol parsing into high-accuracy parsing, medium-accuracy parsing and low-accuracy parsing according to the evaluation results; Adjust the processing module for high-accuracy parsing, which is directly used to generate the final protocol format tree; In the logic evaluation module, according to the correlation between the sequence number and the timestamp and the matching between the length field and the subsequent data field, the correlation anomaly index between adjacent fields is calculated to evaluate the confusion of the logical relationship between adjacent fields. The method for obtaining the correlation anomaly index is as follows: Get the length of the data segment defined in the protocol and build a time series ; and the actual data length carried in each protocol data packet, construct time series B ; Use the DTW algorithm to calculate the minimum matching cost between the two time series, determine the matching degree between the length field and the data field, and create a cost matrix D of size n×m, where D(i,j) represents the distance between the i-th element in sequence A and the j-th element in sequence B, with the starting point being D(1,1) and the end point being D(n,m); use the distance function , for each sequence position (i, j), calculate the squared difference distance between the i-th value in sequence A and the j-th value in sequence B, expressed as: ; By recursive formula, calculate the cumulative cost matrix D(i,j), which means from arrive and arrive The minimum matching cost is expressed as: ; Among them, the min function ensures that the cumulative cost of the path is minimized; the recursive formula finds the optimal path through three path directions: from the top, from the left, and from the upper left, and accumulates the current cost; the cumulative cost of the initial point is: ; The cumulative cost of the first row and first column is: ; ; By calculating the final cumulative cost, the matching degree of the two sequences is determined, that is, the matching degree of adjacent fields; the final DTW distance appears in the lower right corner of the matrix D(n,m), and its value represents the total cost of the minimum matching path between sequences A and B : ;Will Normalized to the correlation anomaly index, the expression is: ; In the formula, is the correlation anomaly index.

2. According to claim 1, an automatic industrial control protocol reverse system based on a global voting expert algorithm is characterized in that: The adjustment processing module is also used to perform several iterative calculations through the global voting expert algorithm for low-accuracy parsing and re-derive field boundaries using context information.

3. The automatic industrial control protocol reverse engineering system based on the global voting expert algorithm according to claim 1 is characterized in that: The automatic industrial control protocol reverse system also includes: The anomaly prediction module is used to predict and analyze the degree of anomaly of the protocol parsing accuracy within a fixed time period for medium accuracy parsing. If the degree of anomaly is high, the parameters of the global voting expert algorithm are adjusted. If the degree of anomaly is low, the cluster is locally optimized.

4. The automatic industrial control protocol reverse engineering system based on the global voting expert algorithm according to claim 1 is characterized in that: In the field boundary identification module, a global voting expert algorithm is used to determine the field boundary of each message. The sliding window method is used to move byte by byte in the message to analyze the pattern of each position. In each window, the field boundary is voted on through a global voting mechanism. The voting is based on the high-frequency byte patterns and message structure characteristics in the field. During the movement of the sliding window, the global context information is used to analyze the global pattern in the message, and the boundary information of the local message is compared with the global context. After the voting results are accumulated, the position with the highest number of votes is identified as the candidate point for the field boundary. The boundary identified by the global voting expert algorithm is inferred to determine whether it is accurate, and the candidate field boundary is verified. If the boundary point appears repeatedly in multiple messages and the number of votes is higher than any boundary, it is the boundary of a fixed field.

5. The automatic industrial control protocol reverse engineering system based on global voting expert algorithm according to claim 1 is characterized by: In the message clustering evaluation module, the dominant function code ratio variation index is generated according to the frequency of occurrence of each function code in the cluster to evaluate the diversity of function codes in the same cluster. The dominant function code ratio variation index is obtained as follows: Mark the function code distribution in the cluster as ,in, Indicates function code Frequency of occurrence in the cluster, dominant function code is the function code with the highest frequency; calculate the total entropy of the function code distribution in the cluster , the expression is: ; k is the number of different function codes in the cluster, and the dominant function code is calculated , the expression is: ; Calculate the entropy difference between the total entropy and the entropy of the dominant function code , the expression is: ; The entropy difference represents the difference between the overall diversity of the function code and the concentration of the dominant function code. The dominant function code ratio variation index is calculated, and the expression is: ; In the formula, is the dominant function code ratio variation index.

6. The automatic industrial control protocol reverse engineering system based on the global voting expert algorithm according to claim 5 is characterized by: In the parsing accuracy evaluation module, the accuracy of protocol parsing is evaluated based on the confusion of logical relationships between adjacent fields and the diversity of function codes in the same cluster; The correlation anomaly index and the dominant function code ratio variation index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the accuracy value label of the protocol parsing for each group of first eigenvectors as the prediction target, and minimizes the sum of the prediction errors of the accuracy value labels of all protocol parsing as the training target. The machine learning model is trained until the sum of the prediction errors converges, and the model training is stopped. The accuracy value of the protocol parsing is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

7. The automatic industrial control protocol reverse engineering system based on the global voting expert algorithm according to claim 6 is characterized by: The acquired accuracy value of the protocol analysis is compared with a gradient standard threshold, wherein the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the accuracy value of the protocol analysis is compared with the first standard threshold and the second standard threshold respectively; If the accuracy value of the protocol analysis is greater than the second standard threshold, it is classified as a high-accuracy analysis and a high-accuracy analysis signal is generated; if the accuracy value of the protocol analysis is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is classified as a medium-accuracy analysis and a medium-accuracy analysis signal is generated; if the accuracy value of the protocol analysis is less than the first standard threshold, it is classified as a low-accuracy analysis and a low-accuracy analysis signal is generated.

8. The automatic industrial control protocol reverse engineering system based on global voting expert algorithm according to claim 1 is characterized by: In the anomaly prediction module, for medium accuracy parsing, the degree of anomaly of the protocol parsing accuracy within a fixed time period is predicted and analyzed; When the accuracy value of the protocol analysis generated within a fixed time period is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is classified as medium accuracy analysis, and the accuracy values ​​of the protocol analysis generated in subsequent fixed time periods are collected, and a data set is established. The mean and standard deviation of the data set are calculated, and after analysis, the abnormal degree of the protocol analysis accuracy in subsequent fixed time periods is predicted.

9. The automatic industrial control protocol reverse engineering system based on the global voting expert algorithm according to claim 8 is characterized by: If the mean accuracy value in the data set is greater than or equal to the reference threshold of the mean accuracy value, and the standard deviation of the accuracy value is less than the reference threshold of the standard deviation of the accuracy value, it means that the analysis result is stable and the accuracy of the analysis is high. At this time, no warning signal is generated, the abnormality is low, and the cluster is locally optimized; If the mean of the accuracy values ​​is greater than or equal to the reference threshold of the mean of the accuracy values, and the standard deviation of the accuracy values ​​is greater than or equal to the reference threshold of the standard deviation of the accuracy values, it indicates that there is a local parsing error, and a warning signal is generated at this time, the abnormality is high, and the parameters of the global voting expert algorithm are adjusted; If the mean of the accuracy values ​​is less than the reference threshold of the mean of the accuracy values, and the standard deviation of the accuracy values ​​is greater than or equal to the reference threshold of the standard deviation of the accuracy values, the accuracy value of the protocol analysis is low and the distribution is unstable. At this time, a warning signal is generated, the abnormality is high, and comprehensive adjustments are made; If the mean accuracy value is less than the reference threshold of the mean accuracy value, and the standard deviation of the accuracy value is less than the reference threshold of the standard deviation of the accuracy value, the accuracy value of the protocol analysis is low but the distribution is stable, and no warning signal is generated at this time, and the abnormality is low; Perform local optimization.

Images