A private network protocol classification method based on state machine subgraph isomorphism matching

Abstract

The present invention discloses a private protocol classification method based on isomorphic matching of protocol state machine subgraphs, which includes a private protocol state machine generation stage and a state machine matching classification stage. In the private protocol state machine generation stage, the captured device network traffic is first parsed to extract the binary sequence of the protocol application layer, and then the alignment algorithm is used to parse the protocol field content from the binary sequence, and the clustering algorithm is further used to merge similar fields to obtain the private protocol message type and message format, and finally the augmented prefix tree structure is used to merge the states from the device interaction messages to generate the protocol state machine; in the state machine matching classification stage, the subgraph isomorphic matching algorithm is used to match the state machine diagram with the known standard protocol state machine to achieve rapid classification of private protocols. The present invention can help device protocol analysts quickly locate the standard protocols that network equipment manufacturers refer to when designing their own self-defined private protocols.

Description

Technical Field

[0001] The invention belongs to the field of network security, and in particular relates to a private protocol identification method based on isomorphic matching of protocol state machine subgraphs. Background Art

[0002] The future industrial Internet is a new industrial ecology, key infrastructure and new application model that deeply integrates the new generation of information and communication network technology with industrial manufacturing. Through the safe and reliable intelligent connection of man, machine and things, it realizes the comprehensive connection of all production factors, the entire industrial chain and the entire value chain, promotes fundamental changes in the production mode and enterprise form of the manufacturing industry, forms a new industrial production and service system, and significantly improves the digitalization, networking and intelligent development level of the manufacturing industry. However, its security issues are gradually exposed. The increasingly frequent industrial Internet security incidents show that there are already a large number of security vulnerabilities and hidden dangers in the industrial Internet. In order to reduce the occurrence of industrial control system security incidents in the future and make the data transmission of industrial control systems in the communication process safer, it is of great significance to study the possible security issues of the transmission protocol used in the industrial control system, dig out the possible vulnerabilities in the industrial control protocol, and then maintain the security of the industrial control system.

[0003] Based on commerciality and security, most protocols used in the Industrial Internet do not have public standard documents. Non-standard unknown private industrial control protocols have brought huge challenges to the network behavior management, intrusion detection, fuzz testing, etc. of the Industrial Internet. In order to conduct security research on these protocols, it is necessary to obtain relevant knowledge of these protocols, and a common method is protocol reverse engineering.

[0004] A typical example of protocol reverse engineering technology is the Protocol Informatics project initiated by Marshall Beddoe in 2004. The PI project searches for sequences with special significance from a large amount of network data streams and obtains the protocol structure through comparative analysis of a large amount of network data. Based on the PI project, scholars have successively conducted a lot of research. Leita et al.'s Honeyd configuration script automatic extraction tool ScriptGen is based on the algorithm flow of the PI project. Another significant example of the message sequence analysis method is the protocol reverse engineering scheme Discoverer proposed by Cui et al. This scheme uses domains as primitives for sequence alignment, which is more targeted. After that, with the continuous maturity of artificial intelligence technologies such as natural language processing and machine learning, this type of protocol reverse analysis technology has also developed rapidly. Not only has the research object expanded from text protocols to binary protocols, but the types of methods have also been continuously enriched, such as NetZob, MSA-HMM and other methods based on sequence alignment, Biprominer, ProGraph, HsMM and other methods based on probability models, AutoReEngine method based on frequent sets, FieldHunter and other methods based on semantic analysis. In recent years, new methods have been proposed, such as Netplier, which uses a protocol keyword probability extraction algorithm based on an intuitive measurement of clustering effects.

[0005] Over the past few years, there has been a growing interest in graphs among the scientific community active in the fields of pattern analysis, pattern recognition, and computer vision, and the applications using graphs have multiplied. Graphs are often used to provide a structural description of an image by decomposing it into its parts and associating the nodes and edges of the graph to the components and their relationships. From the perspective of pattern recognition, the most important problem in graph processing is matching graphs or subgraph comparison. Subgraph isomorphism has always been a very important problem in graph theory, and several related papers have been proposed to solve this problem over the years. Given two graphs G1 = (V1, E1), G2 = (V2, E2) and a mapping G1 and G2 are isomorphic if and only if M is a two-color and the corresponding edge is also a two-color. If a subgraph of G1 and a bicolor of G2 are isomorphic to the subgraphs of G1 and G2, then the subgraphs of G1 and G2 are said to be isomorphic. Among them, the backtracking algorithm proposed by Ullmann is the earliest algorithm that can find subgraph isomorphisms, which can significantly reduce the size of the search space. Others include the Nauty algorithm, the SD algorithm, the VF algorithm, and the VF2 algorithm that improves the VF algorithm.

[0006] Existing non-standard unknown private industrial control protocols are generally improved from more mature public protocols. Therefore, if it is possible to identify what kind of public protocol the unknown protocol is, the efficiency of protocol security research will be improved. Existing methods cannot quickly and effectively identify what kind of public protocol the unknown protocol is. In response to the above problems, the present invention utilizes the highly similar graph structure between the state machines of the protocol to be analyzed and the referenced standard protocol, that is, the state machine obtained by reverse analysis from the captured traffic is only a sub-state machine of the original protocol, and proposes a private protocol classification method based on isomorphic matching of protocol state machine subgraphs. This patent uses subgraph isomorphic algorithm matching to complete protocol identification. Summary of the invention

[0007] Purpose of the invention: The technical problem to be solved by the present invention is how to help device protocol analysts quickly locate the standard protocols that network equipment manufacturers define when designing their own private protocols, thereby providing assistance for further analysis of the protocol. The present invention only analyzes the captured private protocol traffic to identify the protocol type problem, and proposes a private network protocol classification method based on isomorphic matching of state machine subgraphs.

[0008] The above purpose is achieved through the following technical solutions:

[0009] A private network protocol classification method based on state machine subgraph isomorphic matching, the method includes a private protocol state machine generation stage and a state machine matching classification stage, and the specific steps are as follows:

[0010] Phase 1: Protocol state machine generation phase

[0011] Step 1: Message alignment

[0012] Step 1.1: Parse the captured device network traffic and extract the binary sequence of the protocol application layer;

[0013] Step 1.2: Use the sequence alignment algorithm to parse the protocol field content from the binary sequence, and obtain each aligned protocol similar field as the key field candidate field;

[0014] Step 2: Message clustering

[0015] Use clustering algorithm to merge similar fields to obtain private protocol message type and message format;

[0016] Step 2.1: Create a matching score table

[0017] First, for each message sequence, calculate the matching score with other message sequences using the method in step 1.2, and create a matching score table for each message sequence;

[0018] Step 2.2: Hierarchical clustering

[0019] The message sequence with the highest score is selected from the matching score table, and the matching scores of the two sequences are obtained. The two messages are classified into one category. According to this method, the triples of each message sequence are then detected and clustered using a hierarchical clustering algorithm;

[0020] Step 3: Extract key fields

[0021] Key field extraction uses probability distribution statistics to count the distribution of each field in the class generated by each cluster, calculates the distribution variance, and determines the smallest one as the key field of the message, which is used to identify the message type generated by this cluster;

[0022] Step 4: Protocol state machine inference

[0023] The purpose of state machine inference is to infer a finite state machine (FSM) that can identify different message categories and fully describe the various states of the protocol session as well as the conditions and paths of state transitions.

[0024] Step 4.1: Construct an augmented prefix tree

[0025] According to the results of message clustering and key field extraction, the messages in the input session set are sorted according to message sequence and message type to construct an augmented prefix tree (APTA)

[0026] The augmented prefix tree is a finite automaton that cannot be completely determined. The state transition diagram is represented by a tree. The construction of the augmented prefix tree starts from the initial state and separates the data stream according to different sessions. Each session represents a branch of the tree, and the same branch is used for overlapping messages in two sessions. In the augmented prefix tree, all states are marked as empty by default, which is prone to over-merging in the subsequent state merging process, so each state must be marked.

[0027] Step 4.2: Status Mark

[0028] The purpose of state tagging is to avoid over-merging of states. State tagging extracts a prerequisite model from the observed session, which is represented by a regular expression as shown in the following formula:

[0029] .*(r1|…|r k )(a1|…|a j )*,(r,a1,…,a j ∈M)

[0030] The "." in the formula represents any type of message, and the "*" represents any number of repetitions of the previous part. The state with such a model is called a prerequisite. k Indicates the type of message to be received, a1,…,a j Indicates the message types that can be received after r. The prerequisite means that the server only needs to receive r1,…,r before executing the message sequence q. k Any type of message in can then reach the state of accepting message sequence q;

[0031] Once the prerequisites for all message sequences are expressed, all states in the augmented prefix tree must be labeled. The label of each state is the set of all allowed input information types for this state.

[0032] Step 4.3: State machine simplification

[0033] Phase 2: State machine matching and classification phase

[0034] The subgraph isomorphism problem can be described as follows: given two graphs G1 = (V1, E1), G2 = (V2, E2) and a mapping G1 and G2 are said to be isomorphic if and only if M is a two-color and the corresponding edge is also a two-color, V1 represents the set of points in G1, V2 represents the set of points in G2, E1 represents the set of edges in G1, and E2 represents the set of edges in G2. If is a subgraph of G1 and a bicolor of G2, then the subgraphs of G1 and G2 are said to be isomorphic;

[0035] The subgraph isomorphism matching algorithm adopts the state space representation (SSR) method of the matching process, allowing the grammatical and semantic comparison of the node pairs to be matched at the same time, and using feasibility rules to prune the search tree. The state space expression consists of the state equation and the output equation. It is a formula that fully describes the control system in the state space. It can reflect the changes of all independent variables of the system, so that all internal motion states of the system can be determined at the same time, and the initial conditions can be easily handled. Using the state space expression, each state s of the matching process can be associated with a partial mapping M(s), which is a subset of the final mapping M. The two graphs G1=(V1,E1) and G2=(V2,E2) for subgraph isomorphism matching are called G1(s) and G2(s) by the nodes associated with this partial mapping M(s). According to the definition in the algorithm, the transition from state s to the successor state s' is to add a pair of matching nodes (n,m) to the partial graph associated with the current state s in the state space expression, and further perform consistency checks on the successor states to generate s'. The algorithm introduces a set of rules that can verify the consistency test, which can prove that in the case of isomorphism or graph subgraph isomorphism, the partial graphs G1(s) and G2(s) related to M(s) after passing the consistency condition test are isomorphic. In addition, by adding a set of K-look-ahead rules, that is, if the consistent state s has no consistent subsequent state after k steps, the number of states generated in the process can be further reduced, so that the algorithm can significantly reduce the time complexity of the algorithm.

[0036] Step 1: Convert the protocol state machine obtained in the first phase into a directed graph

[0037] Step 2: Use the subgraph isomorphism algorithm to match the standard state machine of a protocol. If the match is successful, it means that the protocol state machine is a subgraph isomorphic to the standard state machine, that is, the captured protocol belongs to this protocol.

[0038] Step 2.1: The graphs to be verified for isomorphism or subgraph isomorphism are G1 and G2, the initial state of the algorithm is s0, and the initial isomorphism mapping M(s0) is an empty set.

[0039] Step 2.2: Determine whether the current state s contains all the points in G2. The state s indicates that the current partial mapping M(s) satisfies the isomorphism. If the associated mapping M(s) of the current state s contains all the points in G2, then output M(s) and the algorithm ends; otherwise, go to step 2.3.

[0040] Step 2.3: Calculate the candidate point pair set P(s) of the graphs G1 and G2 to be verified isomorphic, and select point pairs (n, m) from P(s), n∈G1, m∈G2,

[0041] Step 2.4: The rule traverses the point pairs in P(s), and determines whether the point pair (n, m) is added to the mapping and updates s to s' by checking whether the feasibility rule F(s,n,m) of the current state is true.

[0042] (1) If the return value is true, it means that after adding the point pair (n, m) to the current state s, the partial mapping M(s') of the new state s' still satisfies the isomorphism;

[0043] (2) If the return value of F(s,n,m) is false, it means that (n,m) should not be added to the current state, which can play a role in pruning;

[0044] The consistency check function F(s,n,m) is implemented by giving forward node check rules, successor node check rules, node addition rules, node deletion rules and new state check rules;

[0045] After traversing the point pairs in P(s), jump to step 2.2; in step 2.2, determine whether the algorithm ends or continues; verify whether G1 and G2 are isomorphic or subgraph isomorphic based on the fact that the associated mapping M(s) of the final state s contains all the points in G2.

[0046] Furthermore, the basic idea of ​​the sequence alignment algorithm described in step 1.2 of phase 1 for alignment and similarity calculation is: use the score to determine the matching degree of the two input strings, if the two characters at the corresponding positions are different, a negative score is given, if the two characters are the same, a positive score is given, and finally the total score of the two inputs is calculated;

[0047] The scoring rules for matching detection using the sequence alignment algorithm are as follows. The algorithm defines three cases: MATCH, DISMATCH and INDEL:

[0048] ⑴ Two basic domains match (MATCH), and get positive scores;

[0049] ⑵ The two basic domains do not match (DISMATCH), resulting in negative scores;

[0050] ⑶ If the element at the corresponding position of a sequence is empty (INDEL), it will also get a negative score.

[0051] Based on the penalty rule, a scoring matrix is ​​constructed using the following formula:

[0052]

[0053] Align and cluster the unknown protocol messages so that the messages of the same type are classified into the same category, so as to carry out the next step of reversing the state machine according to the message sequence;

[0054] Furthermore, the hierarchical clustering described in step 2.2 of phase 1 is performed by pairwise comparison clustering. To measure the distance between clusters generated by clustering, it is necessary to determine the similarity measurement criterion. Assume that C i , C j If there are two clusters, then according to the similarity metric, C i , C j The distance between them is:

[0055]

[0056] Step 2.2.1: Initialization

[0057] In the initial stage, each sequence is divided into one category and each is a leaf node of a tree.

[0058] Step 2.2.2: Generate a system spanning tree

[0059] The two closest sequences calculated according to the similarity metric are merged to define a new node. This process is repeated until all sequences are added, and finally a system spanning tree is obtained.

[0060] Furthermore, the state machine simplification method described in step 4.3 of phase 1 is to simplify the state on the red and blue node framework. The purpose of marking the state as red and blue nodes is to determine the order of merging. That is, all nodes are divided into two categories: red nodes and blue nodes. The specific simplification steps are as follows:

[0061] ⑴ First, start traversing from the root node of APTA, mark all root nodes in red, mark all child nodes of the root node in blue, and mark other nodes as unknown nodes;

[0062] ⑵Try to merge a red node and a blue node. If the labels added to the two nodes in the state marking phase are the same, they can be merged. When merging, first traverse all subtrees of the blue node and add them to the subtree collection of the red node. All newly added subtrees are regarded as candidates and marked as blue. If the merge fails due to different labels, the blue node is promoted to a red node and all its subtrees are marked as blue.

[0063] ⑶ For all blue nodes that have not been merged, repeat steps 1 and 2. During the processing, a blue node needs to be compared with all red nodes to determine whether it can be merged. If it cannot be merged with all red nodes, the blue node is promoted to a red node.

[0064] (4) After all blue nodes are merged, the state transition diagram obtained is the result of state machine simplification, which is a minimized DFA.

[0065] Furthermore, the construction rule of the candidate point pair set P(s) described in step 2.3 of stage 2 is as follows:

[0066] out-terminal collection is the set of successor nodes in G1 that do not belong to the nodes in M1(s), where M1(s) is the part of the mapping M(s) corresponding to state s that is associated with G1. Define the in-terminal set is the set of predecessor nodes that do not belong to M1(s) but belong to the nodes in M1(s); similarly, and M2(s) is the part of the mapping M(s) corresponding to state s that is associated with G2;

[0067] (1) If and None of them are empty. express The node with the smallest label in (any sorting method is acceptable);

[0068] (2) If and are empty, and and are not empty,

[0069] (3) If all four terminal sets are empty, P(s) = V1-M1(s) × {min(V2-M2(s))};

[0070] (4) When there is only one in-terminal set or only one out-terminal set is empty, it can be proved that it is impossible for state s to construct the final isomorphism, so state s does not need to be analyzed further.

[0071] Furthermore, the forward-looking node verification rule described in step 2.4 of phase 2 checks the consistency of the partial solution M(s) by adding the considered candidate (n, m) to the current partial solution M(s).

[0072] Successor node verification rule: checks the consistency of the partial solution M(s) by adding the considered candidate (n, m) to the current partial solution M(s).

[0073] Add node rules: used to prune the search tree during the process of adding point pairs to improve search efficiency.

[0074] Node pruning rule: used to prune the search tree in the process of deleting point pairs to improve search efficiency.

[0075] New state verification rule: used to prune the search tree in the process of verifying the generated new mapping to improve search efficiency.

[0076] The technical solution of the present invention has the following advantages:

[0077] By combining protocol reverse generation of protocol state machine with subgraph isomorphic matching, a private network protocol classification method based on state machine subgraph isomorphic matching is implemented, which can classify and identify private protocols defined by IoT device manufacturers. It can help device protocol analysts quickly locate the standard protocols that network equipment manufacturers refer to when designing their own private protocols, thereby providing assistance for further analysis of the protocol. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a flow chart of the present invention.

[0079] Figure 2 This is an alignment process for constructing a scoring matrix in an embodiment of the present invention.

[0080] Figure 3 In the embodiment of the present invention, a scoring matrix is ​​constructed to obtain a result based on the backtracking path. DETAILED DESCRIPTION

[0081] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] The embodiment of the present invention discloses a private protocol identification method based on isomorphic matching of protocol state machine subgraphs, such as Figure 1 As shown, the method comprises the following steps:

[0083] Phase 1: Protocol state machine generation phase

[0084] Step 1: Message alignment

[0085] Step 1.1: Parse the captured device network traffic and extract the binary sequence of the protocol application layer part.

[0086] Step 1.2: Use a sequence alignment algorithm to parse the protocol field content from the binary sequence, and obtain each aligned protocol similar field as a candidate key field.

[0087] The basic idea of ​​sequence alignment algorithm for alignment and similarity calculation is to use the score to determine the matching degree of two input strings. If the two characters in the corresponding positions are different, a negative score is given, and if the two characters are the same, a positive score is given. Finally, the total score of the two inputs is calculated.

[0088] The scoring rules for matching detection using the sequence alignment algorithm are as follows. The algorithm defines three cases: MATCH, DISMATCH and INDEL:

[0089] ⑴ Two basic domains match (MATCH), score +1;

[0090] (2) The two basic domains do not match (DISMATCH), score -1;

[0091] ⑶If the element at the corresponding position of a sequence is empty (INDEL), the score is also -1

[0092] Based on the penalty rule, a scoring matrix is ​​constructed by the following formula. When two sequences are aligned, a two-dimensional matrix is ​​constructed. First, the 0th row and the 0th column of the matrix are filled with the characters of S1 and S2 respectively. The row and column numbers of the matrix are represented by i and j respectively, and indel, match, and mismatch are the score changes:

[0093]

[0094] For example, sequence S1=GGATCGA and sequence S2=GAATTCAGTTA, "_" represents a space. To construct a two-dimensional matrix, first fill the 0th row and 0th column of the matrix with S1 and S2 respectively. When filling, be sure to reserve two characters, let (1, 1)=0, then add an INDEL penalty to each vertical and horizontal unit compared to the previous unit, and construct the entire scoring matrix through the formula. Finally, start from the lower right corner of the matrix and trace back upward. The path is toward the upper left, that is, MATCH / DISMATCH. The path to the left means that S1 has an INDEL, and the path to the top means that S2 has an INDEL, and use '-' instead. Finally, we get the following: Figure 2 As a result, it is the global optimal solution, and the lower right corner of the matrix represents the similarity between the two sequences. Finally, according to the backtracking path, we can get Figure 3 result.

[0095] Step 2: Message clustering

[0096] This step further utilizes a clustering algorithm to merge similar fields to obtain the private protocol message type and message format.

[0097] Step 2.1: Create a matching score table

[0098] First, for each message sequence, calculate the matching score with other message sequences using the method in step 1.2, and create a matching score table for each message sequence.

[0099] The matching score table is shown in Table 1. Then filter out the message sequence with the highest score from the table to construct a triple i ,S j ,H ij >(i≠j), S j Represents the message sequence S i The message sequence with the highest matching score, H ij is the matching score of the two sequences. From the table, we can see that for message sequence S1, the highest matching score is S3, with a score of 15. Then the triple is<S1,S3,15> , and then detect the triples of each message sequence.

[0100] Table 1 Examples of sequence matching scores

[0101]

[0102] Step 2.2: Hierarchical clustering

[0103] The message sequence with the highest score is selected from the matching score table, and the matching score of the two sequences is obtained, and the two messages are classified into one category. According to this method, the triples of each message sequence are then detected and clustered using a hierarchical clustering algorithm. Hierarchical clustering is performed by pairwise comparison clustering. To measure the distance between clusters generated by clustering, it is necessary to determine the similarity measurement criterion. Assuming that Ci and Cj are two clusters, according to the similarity measurement criterion, the distance between Ci and Cj in the hierarchical clustering algorithm is:

[0104]

[0105] Step 2.2.1: Initialization

[0106] In the initial stage, each sequence is divided into one category and each is a leaf node of a tree.

[0107] Step 2.2.2: Generate a system spanning tree

[0108] The two closest sequences calculated according to the similarity metric are merged to define a new node. This process is repeated until all sequences are added, and finally a system spanning tree is obtained.

[0109] Step 3: Extract key fields

[0110] ​Key field extraction uses probability distribution statistics to count the distribution of each field in the class generated by each cluster, calculates the distribution variance, and determines the smallest one as the key field of the message to identify the message type generated by this type of cluster.

[0111] Step 4: Protocol state machine inference

[0112] The purpose of state machine inference is to infer a finite state machine (FSM) that can recognize different message categories and fully describe the various states of the protocol session as well as the conditions and paths of state transitions.

[0113] Step 4.1: Construct an augmented prefix tree

[0114] According to the results of message clustering and key field extraction, the messages in the input session set are sorted according to message sequence and message type to construct an augmented prefix tree (APTA)

[0115] The augmented prefix tree is a finite automaton that cannot be completely determined. The state transition diagram is represented by a tree. The construction of the augmented prefix tree starts from the initial state and separates the data stream according to different sessions. Each session represents a branch of the tree, and the same branch is used for overlapping messages in two sessions. In the augmented prefix tree, all states are marked as empty by default, which is prone to over-merging in the subsequent state merging process, so each state must be marked.

[0116] Step 4.2: Status Mark

[0117] The purpose of state tagging is to avoid over-merging of states. State tagging extracts a prerequisite model from the observed session, which can be expressed as a regular expression as shown in the following formula:

[0118] .*(r1|…|r k )(a1|…|a j )*

[0119] The "." in the formula represents any type of message, and the "*" represents any number of repetitions of the previous part. The prerequisite means that the server only needs to receive r1, ..., r before executing the message sequence m. k Only after receiving any type of message in the sequence m can the state of accepting message sequence m be reached.

[0120] Once the prerequisites for all message sequences are expressed, all states in the augmented prefix tree need to be labeled. The label of each state is the set of all the allowed input information types for this state.

[0121] Step 4.3: State machine simplification

[0122] The state machine simplification method in this paper is to simplify the state on the red and blue node framework, that is, to divide all nodes into two categories: red nodes and blue nodes. The specific simplification steps are as follows:

[0123] ⑴ First, start traversing from the root node of APTA, mark all root nodes in red, mark all child nodes of the root node in blue, and mark other nodes as unknown nodes;

[0124] ⑵Try to merge a red node and a blue node. If the labels added to the two nodes in the state marking phase are the same, they can be merged. When merging, first traverse all subtrees of the blue node and add them to the subtree collection of the red node. All newly added subtrees are regarded as candidates and marked as blue. If the merge fails due to different labels, the blue node is promoted to a red node and all its subtrees are marked as blue.

[0125] ⑶ For all blue nodes that have not been merged, repeat steps 1 and 2. During the processing, a blue node needs to be compared with all red nodes to determine whether it can be merged. If it cannot be merged with all red nodes, the blue node is promoted to a red node.

[0126] (4) After all blue nodes are merged, the state transition diagram obtained is the result of state machine simplification, which is a minimized DFA.

[0127] Phase 2: State machine matching and classification phase

[0128] The subgraph isomorphism problem can be described as follows: given two graphs G1 = (V1, E1), G2 = (V2, E2) and a mapping G1 and G2 are isomorphic if and only if M is a two-color and the corresponding edge is also a two-color. is a subgraph of G1 and a bicolor of G2, then the subgraphs of G1 and G2 are said to be isomorphic.

[0129] The subgraph isomorphism matching algorithm adopts the state space representation (SSR) method of the matching process, allowing the grammatical and semantic comparison of the node pairs to be matched at the same time, and using feasibility rules to prune the search tree. The state space expression consists of the state equation and the output equation. It is a formula that fully describes the control system in the state space. It can reflect the changes of all independent variables of the system, so that all internal motion states of the system can be determined at the same time, and the initial conditions can be easily handled. Using the state space expression, each state s of the matching process can be associated with a partial mapping M(s), which is a subset of the final mapping M. The two graphs G1=(V1,E1) and G2=(V2,E2) for subgraph isomorphism matching are called G1(s) and G2(s) by the nodes associated with this partial mapping M(s). According to the definition in the algorithm, the transition from state s to the successor state s' is to add a pair of matching nodes (n,m) to the partial graph associated with the current state s in the state space expression, and further perform consistency checks on the successor states to generate s'. The algorithm introduces a set of rules that can verify the consistency test, which can prove that in the case of isomorphism or graph subgraph isomorphism, the partial graphs G1(s) and G2(s) related to M(s) after passing the consistency condition test are isomorphic. In addition, by adding a set of K-look-ahead rules, that is, if the consistent state s has no consistent subsequent state after k steps, the number of states generated in the process can be further reduced, so that the algorithm can significantly reduce the time complexity of the algorithm.

[0130] Step 1: Convert the protocol state machine obtained in the first phase into a directed graph

[0131] Step 2: Use the subgraph isomorphism algorithm to match the standard state machine of a protocol. If the match is successful, it means that the protocol state machine is a subgraph isomorphic to the standard state machine, that is, the captured protocol belongs to this protocol.

[0132] Step 2.1: The graphs to be verified for isomorphism or subgraph isomorphism are G1 and G2, the initial state of the algorithm is s0, and the initial isomorphism mapping M(s0) is an empty set.

[0133] Step 2.2: Determine whether the current state s contains all points in G2. The state s indicates that the current partial mapping M(s) satisfies isomorphism. If the associated mapping M(s) of the current state s contains all points in G2, then output M(s) and the algorithm ends. Otherwise, go to step 2.3.

[0134] Step 2.3: Calculate the candidate point pair set P(s) of the graphs G1 and G2 to be verified to be isomorphic, and select point pairs (n, m) from P(s), n∈G1, m∈G2.

[0135] The construction rules of the candidate point pair set P(s) are as follows:

[0136] out-terminal collection It is the set of successor nodes in G1 that are not in M1(s). Define the in-terminal set is the set of predecessor nodes that do not belong to M1(s) but belong to the nodes in M1(s). Similarly, we define and

[0137] (5) If and None of them are empty. express The node with the smallest label in (either sorting method is acceptable).

[0138] (6) If and are empty, and and are not empty,

[0139] (7) If all four terminal sets are empty, P(s) = V1-M1(s)×{min(V2-M2(s))}.

[0140] (8) When there is only one in-terminal set or only one out-terminal set is empty, it can be proved that state s cannot construct the final isomorphism, so state s does not need to be analyzed further.

[0141] Step 2.4: The rule traverses the point pairs in P(s), and determines whether the point pair (n, m) is added to the mapping and updates s to s' by checking whether the current state feasibility rule F(s,n,m) is true.

[0142] (3) If the return value is true, it means that after adding the point pair (n, m) to the current state s, the partial mapping M(s') of the new state s' still satisfies the isomorphism;

[0143] (4) If the return value of F(s,n,m) is false, it means that (n,m) should not be added to the current state, which can play a role in pruning.

[0144] The consistency check function F(s,n,m) is implemented by giving forward node checking rules, successor node checking rules, adding node rules, deleting node rules and new state checking rules.

[0145] Look-ahead node verification rule: checks the consistency of the partial solution M(s) by adding the considered candidate (n, m) to the current partial solution M(s).

[0146] Successor node verification rule: checks the consistency of the partial solution M(s) by adding the considered candidate (n, m) to the current partial solution M(s).

[0147] Add node rules: used to prune the search tree during the process of adding point pairs to improve search efficiency.

[0148] Node pruning rule: used to prune the search tree in the process of deleting point pairs to improve search efficiency.

[0149] New state verification rules: used to prune the search tree in the process of verifying the generated new mapping to improve search efficiency.

[0150] The forward node inspection rule, the subsequent node inspection rule, the node addition rule, the node deletion rule and the new state inspection rule are denoted as R pred , R succ , R in , R out and R new , the specific implementation is as follows:

[0151] remember Similarly, T2(s) and Then there are:

[0152]

[0153]

[0154]

[0155]

[0156]

[0157] After traversing all the point pairs in P(s), jump to step 2.2. In step 2.2, determine whether the algorithm ends or continues. Verify whether G1 and G2 are isomorphic or subgraph isomorphic based on the fact that the associated mapping M(s) of the final state s contains all the points in G2.

Claims

1. A private network protocol classification method based on state machine subgraph isomorphic matching, characterized in that: The method comprises the following steps: Phase 1: Protocol state machine generation phase Step 1: Message alignment Step 1.1: Parse the captured device network traffic and extract the binary sequence of the protocol application layer; Step 1.2: Use the sequence alignment algorithm to parse the protocol field content from the binary sequence, and obtain each aligned protocol similar field as the key field candidate field; The basic idea of ​​the sequence alignment algorithm described in step 1.2 of phase 1 for alignment and similarity calculation is to use a score to determine the matching degree of two input strings. If the two characters at the corresponding positions are different, a negative score is given. If the two characters are the same, a positive score is given. Finally, the total score of the two inputs is calculated. The scoring rules for matching detection using the sequence alignment algorithm are as follows. The algorithm defines three cases: MATCH, DISMATCH and INDEL: ⑴ Two basic domains match (MATCH), and get positive scores; ⑵ The two basic domains do not match (DISMATCH), resulting in negative scores; ⑶ If the element at the corresponding position of a sequence is empty (INDEL), it will also get a negative score. Based on the penalty rule, a scoring matrix is ​​constructed using the following formula: Align and cluster the unknown protocol messages so that the messages of the same type are classified into the same category, so as to carry out the next step of reversing the state machine according to the message sequence; Step 2: Message clustering Use clustering algorithm to merge similar fields to obtain private protocol message type and message format; Step 2.1: Create a matching score table First, for each message sequence, calculate the matching score with other message sequences using the method in step 1.2, and create a matching score table for each message sequence; Step 2.2: Hierarchical clustering Filter the message sequence with the highest score from the matching score table, calculate the matching score of the two sequences, classify the two messages into one category, and then detect the triples of each message sequence and cluster them using the hierarchical clustering algorithm; The hierarchical clustering described in step 2.2 of phase 1 is performed by pairwise comparison clustering. To measure the distance between clusters generated by clustering, it is necessary to determine the similarity measurement criterion. Assume that C i , C j If there are two clusters, then according to the similarity metric, C i , C j The distance between them is: Step 2.2.1: Initialization In the initial stage, each sequence is divided into one category and each is a leaf node of a tree; Step 2.2.2: Generate a system spanning tree Merge the two closest sequences calculated according to the similarity metric to define a new node. Repeat this process until all sequences are added, and finally get a phylogenetic tree. Step 3: Extract key fields Key field extraction uses probability distribution statistics to count the distribution of each field in the class generated by each cluster, calculates the distribution variance, and determines the smallest one as the key field of the message, which is used to identify the message type generated by this cluster; Step 4: Protocol state machine inference The purpose of state machine inference is to infer a finite automaton that can identify different message categories and fully describe the various states of the protocol session as well as the conditions and paths of state transitions; Step 4.1: Construct an augmented prefix tree According to the results of message clustering and key field extraction, the messages in the input session set are sorted according to message sequence and message type to construct an augmented prefix tree; Step 4.2: Status Mark The state tagging extracts the prerequisite model from the observed sessions, which is represented by a regular expression as shown in the following formula: .*(r1|...|r k )(a1|...|a j )*,(r,a1,...,a j ∈M) The "." in the formula represents any type of message, and "*" represents any number of repetitions of the previous part. The state with such a model is called a prerequisite, r1, ..., r k Indicates the type of message to be received, a1, ..., a j indicates the type of messages that can be received after r, and the prerequisite indicates that the server only needs to receive r1, ..., r before executing the message sequence q. k Any type of message in can then reach the state of accepting message sequence q; Once the prerequisites for all message sequences are expressed, all states in the augmented prefix tree must be labeled. The label of each state is the set of all allowed input information types for this state. Step 4.3: State machine simplification The state machine simplification method described in step 4.3 of phase 1 is to simplify the state on the red and blue node framework. The purpose of marking the state as red and blue nodes is to determine the order of merging, that is, to divide all nodes into two categories: red nodes and blue nodes. The specific simplification steps are as follows: (1) First, start traversing from the root node of APTA, mark all root nodes in red, mark all child nodes of the root node in blue, and mark other nodes as unknown nodes; (2) Try to merge a red node and a blue node. If the labels added to the two nodes in the state marking phase are the same, they can be merged. When merging, first traverse all subtrees of the blue node and add them to the subtree collection of the red node. The newly added subtrees are all regarded as candidates and marked as blue. If the merge fails due to different labels, the blue node is promoted to a red node and all its subtrees are marked as blue. (3) Repeat steps (1) and (2) for all blue nodes that have not been merged. During the processing, a blue node needs to be compared with all red nodes to determine whether it can be merged. If it cannot be merged with all red nodes, the blue node is promoted to a red node. (4) After all blue nodes are merged, the state transition diagram obtained is the result of state machine simplification, which is a minimized DFA; Phase 2: State machine matching and classification phase The subgraph isomorphism problem can be described as follows: given two graphs G1 = (V1, E1), G2 = (V2, E2) and a mapping G1 and G2 are said to be isomorphic if and only if M is a two-color and the corresponding edge is also a two-color, V1 represents the set of points in G1, V2 represents the set of points in G2, E1 represents the set of edges in G1, and E2 represents the set of edges in G2. If is a subgraph of G1 and a bicolor of G2, then the subgraphs of G1 and G2 are said to be isomorphic; Step 1: Convert the protocol state machine obtained in the first phase into a directed graph Step 2: Use the subgraph isomorphism algorithm to match the standard state machine of a protocol. If the match is successful, it means that the protocol state machine is a subgraph isomorphic to the standard state machine, that is, the captured protocol belongs to this protocol. Step 2.1: The graphs to be verified for isomorphism or subgraph isomorphism are G1 and G2, the initial state of the algorithm is s0, and the initial isomorphism mapping M(s0) is an empty set; Step 2.2: Determine whether the current state s contains all points in G2. The state s indicates that the current partial mapping M(s) satisfies isomorphism. If the associated mapping M(s) of the current state s contains all points in G2, then output M(s) and the algorithm ends; otherwise, go to step 2.3; Step 2.3: Calculate the candidate point pair set P(s) of the graphs G1 and G2 to be verified isomorphic, and select point pairs (n, m) from P(s), n∈G1, m∈G2, The construction rules of the candidate point pair set P(s) described in step 2.3 of stage 2 are as follows: out-terminal collection is the set of successor nodes in G1 that do not belong to the nodes in M1(s), where M1(s) is the part of the mapping M(s) corresponding to state s that is associated with G1. Define the in-terminal set is the set of predecessor nodes that do not belong to M1(s) but belong to the nodes in M1(s); similarly, and M2(s) is the part of the mapping M(s) corresponding to state s that is associated with G2; (1) If and None of them are empty. express The node with the smallest label in ; (2) If and are empty, and and are not empty, (3) If all four terminal sets are empty, P(s) = V1-M1(s) × {min(V2-M2(s))}; When there is only one in-terminal set or only one out-terminal set is empty, it can be proved that it is impossible for state s to construct the final isomorphism, so state s does not need to be analyzed further; Step 2.4: The rule traverses the point pairs in P(s), and determines whether the point pair (n, m) is added to the mapping and updates s to s' by checking whether the feasibility rule F(s, n, m) of the current state is true. (1) If the return value is true, it means that after adding the point pair (n, m) to the current state s, the partial mapping M(s') of the new state s' still satisfies the isomorphism; (2) If the return value of F(s, n, m) is false, it means that (n, m) should not be added to the current state, which can play a role in pruning; The consistency check function F(s, n, m) is implemented by giving forward node check rules, successor node check rules, node addition rules, node deletion rules and new state check rules; After traversing the point pairs in P(s), jump to step 2.2; in step 2.2, determine whether the algorithm ends or continues; verify whether G1 and G2 are isomorphic or subgraph isomorphic based on the fact that the associated mapping M(s) of the final state s contains all the points in G2.

2. According to claim 1, a private network protocol classification method based on state machine subgraph isomorphic matching is characterized in that: The look-ahead node verification rule described in step 2.4 of phase 2: checks the consistency of the partial solution M(s) by adding the considered candidate (n, m) to the current partial solution M(s); Successor node verification rule: checks the consistency of the partial solution M(s) by adding the considered candidate (n, m) to the current partial solution M(s); Add node rule: used to prune the search tree during the process of adding point pairs; Node deletion rule: used to prune the search tree in the process of deleting pairs; New state verification rules: used to prune the search tree during the verification of the generated new mapping.

Images