Protocol message field division method based on binary instruction sequence semantic similarity

Abstract

The invention discloses a protocol message field division method based on binary instruction sequence semantic similarity, which relates to the field of protocol reversion and comprises the following steps of: constructing an instruction set related to a message field based on binary instruction execution information of dynamic taint analysis; based on the instruction set, analyzing use characteristics of message fields in the instruction, and preliminarily dividing the message to obtain candidate fields; and updating and obtaining a final message field division result based on the semantic behavior similarity between the instructions. According to the method, the protocol message fields can be efficiently and accurately divided, the analysis efficiency and accuracy of protocol message semantic inference can be improved, and therefore important support is provided for accurate reverse of protocol messages.

Description

Technical Field

[0001] The present invention relates to the field of protocol reversal, and more specifically to the inference of protocol message field division, in particular to a protocol message field division scheme based on execution traces. Background Art

[0002] Protocol Reverse Engineering (PRE) is a technology that infers protocol specification description information when the protocol specification is not public or available. PRE has important applications in network auditing, software security analysis, security vulnerability mining and other fields involving network management and network attack and defense. Protocol format extraction is the core task of PRE, which aims to infer the syntax and semantics of protocol message fields. Among them, field syntax inference, also known as message field division, is mainly used to determine the structural rules of protocol messages, that is, the starting and ending positions of each message field in the entire message structure. Message field division is the first step and key to protocol message format extraction, and the accuracy of the result directly affects the quality of the entire protocol reverse process.

[0003] Message field division can be done by network traffic-based methods and execution trace tracking-based methods. Network traffic-based methods usually use statistical model building or machine learning techniques. Although they have the advantage of not being restricted by platforms, the accuracy of field division results is greatly affected by data diversity. The execution trace tracking-based method infers the division of protocol fields by monitoring and analyzing the execution trace of the protocol program when parsing the message. Among them, the method based on binary instruction analysis is the mainstream technology in this field. This method does not need to rely on the source code of the protocol implementation, and can directly extract the protocol field division information from the execution process of the protocol implementation program, with significant accuracy.

[0004] However, although rich runtime information can be obtained from the execution instructions of the program, due to the diversity of binary instructions caused by the need for various complex professional analysis technologies and the diversity of protocol code implementations, the current field partitioning method based on binary instruction analysis still faces problems such as over-partitioning and under-partitioning of fields, which affects the accuracy of the field partitioning results. How to accurately infer the format of protocol fields from binary instruction execution information is still an unresolved problem. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method for dividing protocol message fields based on the semantic similarity of binary instruction sequences. This method constructs an instruction set by capturing binary instruction execution information, extracts candidate fields based on the taint propagation of operands in each instruction, and then merges adjacent candidate fields according to the semantic similarity of the instruction sequence to obtain the final field division result. This method improves the efficiency and accuracy of field division in protocol reverse analysis, and provides strong technical support for protocol security assessment.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for dividing protocol message fields based on semantic similarity of binary instruction sequences comprises the following steps:

[0008] Step 1: Send the message msg to be divided into fields to the server with the function of recording instruction execution information, track the taint propagation in bytes, and extract the binary instruction execution information set INS for processing msg from the server's log information;

[0009] Step 2: Based on the binary instruction execution information set INS of msg, according to the usage characteristics of the operands in the instructions, the polluted byte sequence in each instruction is extracted as the candidate field, and the preliminary field division result set L of msg is obtained;

[0010] Step 3: Based on the binary instruction execution information set INS for processing msg and the preliminary field partition result set L of msg, sort the elements in L according to the offset position of the candidate fields from small to large and update L;

[0011] Step 4: Extract the instruction sequence related to each candidate field in L from INS, use the NW algorithm to analyze the semantic similarity of the instruction sequence, merge the adjacent candidate fields with similar semantics in L, and output the final field division result of msg.

[0012] Further, the step 2 specifically includes the following steps:

[0013] Step A1: traverse INS. If INS has not been traversed yet, read an untraversed instruction inst and the operand set OPRD involved in inst from INS, and continue to execute step A2; otherwise, the preliminary field division of msg based on the analysis within the instruction is completed, and the preliminary field division result set L is output;

[0014] Step A2: traverse OPRD. If OPRD has not been traversed yet, read an operand that has not been traversed from OPRD and continue to execute step A3; otherwise, return to step A1;

[0015] Step A3: determine whether operand is contaminated. If so, extract the consecutive byte offset positions in msg of the contaminated operand from the execution information of inst, and add these consecutive byte offset positions as a candidate field to the set L; otherwise, return to step A2;

[0016] Further, the step 4 specifically includes the following steps:

[0017] Step B1: Traverse L in order. If L has not been traversed yet, read the next set of candidate fields L that have not been traversed from L. i and L i+1 , proceed to step B2; otherwise, proceed to step B3;

[0018] Step B2: Determine L i and L i+1 Are the instruction sequences of L similar in semantics? If so, i and L i+1 Merge into a new candidate field and update L i+1 And L i Assign the value to Null; if L i and L i+1 If the instruction sequences are not semantically similar, return to B1;

[0019] Step B3: Delete all null values ​​in L to obtain the final field division result of msg.

[0020] Furthermore, the candidate field L in step B2 is determined i and L i+1 Whether the semantics of the instruction sequences are similar, specifically includes the following steps:

[0021] Step C1: Extract candidate fields L respectively i and L i+1 The corresponding byte sequence bs (i,j) and bs (j+1,k) ;

[0022] Step C2: traverse INS. If INS has not been traversed yet, take out an instruction inst that has not been traversed yet and a byte sequence Seq consisting of the polluted bytes accessed by inst from INS, and continue to execute C3; otherwise, execute step C4;

[0023] Step C3: Determine whether Seq is bs (i,j) and bs (j+1,k) If Seq is bs (i,j) subsequence, then add inst to the candidate field L i Related instruction set I; if Seq is bs(j+1,k) subsequence, then add inst to the candidate field L i+1 Related instruction set J; if Seq is neither bs (i,j) A subsequence of (j+1,k) If the subsequence is , return to step C2;

[0024] Step C4: Calculate the semantic similarity score between set I and set J. If the semantic similarity score is greater than or equal to the similarity threshold, the candidate field L is determined. i and L i+1 is semantically similar; otherwise, the candidate field L is determined i and L i+1 The semantics are not similar.

[0025] It can be seen from the above technical solution that, compared with the prior art, the present invention provides a method for dividing protocol message fields based on the semantic similarity of binary instruction sequences. This method aims at the problem that the current message field division method based on binary instructions has inaccurate division results due to over-division and under-division. In combination with the inter-instruction analysis technology, a method for dividing protocol message fields based on the semantic similarity of binary instruction sequences is proposed. According to the dynamic taint analysis information, this method first extracts the contaminated byte sequence as a candidate field based on the usage characteristics of the operands in the binary instructions, and obtains the preliminary field division preliminary result. Then, the NW algorithm is used to judge the semantic similarity between related instruction sequences and merge fields with similar semantic behaviors, thereby eliminating the over-division and under-division in the preliminary field division results, thereby ensuring the accuracy of the final division results. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flowchart of a method for dividing protocol message fields based on semantic similarity of binary instruction sequences provided by the present invention;

[0027] Figure 2 A schematic diagram of a message msg to be divided into fields and a reference truth value of its field division;

[0028] Figure 3 This is a schematic diagram of the code snippet executed by msg on the server and the corresponding execution information. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The present invention provides a method for dividing protocol message fields based on the semantic similarity of binary instruction sequences, comprising the following steps:

[0031] Step 1: Send the message msg to be divided into fields to the server with the function of recording instruction execution information, track the taint propagation in bytes, and extract the binary instruction execution information set INS for processing msg from the server's log information;

[0032] Step 2: Based on the binary instruction execution information set INS that processes msg, according to the usage characteristics of the operands in the instruction, the taint propagation of msg in each operand is analyzed in bytes, and the polluted byte sequence is extracted as the candidate field to obtain the preliminary field division result set L of msg;

[0033] Step 3: Based on the binary instruction execution information set INS for processing msg and the preliminary field partition result set L of msg, sort the elements in L according to the offset position of the candidate fields from small to large and update L;

[0034] Step 4: Extract the instruction sequence related to each candidate field in L from INS, use the NW algorithm to analyze the semantic similarity of the instruction sequences between adjacent candidate fields in L, merge the adjacent candidate fields with similar instruction sequences in L, and output the final field division result of msg.

[0035] The second step is based on the binary instruction execution information set INS of processing msg, and according to the usage characteristics of the operands in the instruction, the taint propagation of msg in each operand is analyzed in bytes, and the polluted byte sequence is extracted as the candidate field to obtain the preliminary field division result set L of msg, which specifically includes the following steps:

[0036] Step A1: traverse INS. If INS has not been traversed yet, read an untraversed instruction inst and the operand set OPRD involved in inst from INS, and continue to execute step A2; otherwise, the preliminary field division of msg based on the analysis within the instruction is completed, and the preliminary field division result set L is output;

[0037] Step A2: traverse OPRD. If OPRD has not been traversed yet, read an operand that has not been traversed from OPRD and continue to execute step A3; otherwise, return to step A1;

[0038] Step A3: In byte units, determine whether operand is contaminated by some bytes (contamination source) in msg. If it is contaminated by some bytes (b i ,b i+1 ,……,bj ) is polluted, the consecutive byte offset positions (i,i+1,...,j) of the polluted operand are extracted from the execution information of inst, and the consecutive byte offset positions (i,i+1,...,j) are used as a candidate field f (i,j) Add to the set L; otherwise, return to step A2;

[0039] The fourth step extracts the instruction sequence related to each candidate field in L from INS, uses the NW algorithm to analyze the semantic similarity of the instruction sequences between adjacent candidate fields in L, merges the adjacent candidate fields with similar instruction sequences in L, and outputs the final field division result of msg, which specifically includes the following steps:

[0040] Step B1: Traverse L in order. If L has not been traversed yet, read the next set of candidate fields L that have not been traversed and have adjacent field offsets from L. i and L i+1 , that is, f (i,j) and f (j+1,k) , proceed to step B2; otherwise, proceed to step B3;

[0041] Step B2: Determine L i and L i+1 Are the instruction sequences of L similar in semantics? If so, i and L i+1 The two candidate fields f (i,j) and f (j+1,k) Merge into a new candidate field f (i,k) , update L i+1 f (i,k) And L i Assign the value to Null; if L i and L i+1 If the instruction sequences are not semantically similar, return to step B1;

[0042] Step B3: Delete all null values ​​in L to obtain the final field division result of msg.

[0043] The candidate field L in step B2 is judged i and L i+1 Whether the semantics of the instruction sequences are similar, specifically including the following steps:

[0044] Step C1: Extract candidate fields L respectively i and L i+1 , that is, f (i,j) and f (j+1,k) , the corresponding byte sequence bs (i,j) and bs (j+1,k) ;

[0045] Step C2: traverse INS. If INS has not been traversed yet, take out an instruction inst that has not been traversed yet from INS and the byte sequence Seq consisting of the polluted bytes accessed by inst, recorded as bs (p,q) , continue to execute C3; otherwise, execute step C4;

[0046] Step C3: Determine whether Seq is bs (i,j) and bs (j+1,k) If Seq is bs (i,j) If p is greater than or equal to i and q is less than or equal to j, then inst is added to the candidate field L. i Related instruction set I; if Seq is bs (j+1,k) If p is greater than or equal to j+1 and q is less than or equal to k, then inst is added to the candidate field L. i+1 Related instruction set J; if Seq is neither bs (i,j) A subsequence of (j+1,k) If the subsequence is , return to step C2;

[0047] Step C4: Take the operator operator of each instruction inst as the semantic feature of the instruction inst, and construct an operator sequence I based on the operators of all instructions in the instruction sets I and J. op and J op , using the NW sequence similarity comparison algorithm, calculate I op and J op If the semantic similarity score is greater than or equal to the similarity threshold, the candidate field L is determined. i and L i+1 is semantically similar; otherwise, the candidate field L is determined i and L i+1 The semantics are not similar.

[0048] Example

[0049] This embodiment discloses a method for dividing protocol message fields based on the semantic similarity of binary instruction sequences. Figure 1 As shown, the following steps are included:

[0050] 1) Send the message msg to be divided into fields to the server with the function of recording instruction execution information, perform taint propagation tracking in bytes, and extract and process the binary instruction execution information set INS of msg.

[0051] By inserting the server binary file, a taint propagation engine is constructed with the input message as the taint source, and the execution information of msg on the server is obtained in bytes, including the taint propagation of the execution instructions and the operands in the instructions. For example, in this example, the request message msg to be divided into fields is sent to the server, and the Figure 2 The execution information set INS of msg on the server side, such as Figure 3 The INS fragment shown contains the instruction sequence related to the execution of some bytes in msg.

[0052] 2) Traverse such as Figure 3 The execution instruction information INS is shown in Figure 1. The operand usage characteristics of each instruction are taken out from INS, and the taint propagation of msg in each operand is analyzed in bytes. Table 1 shows the taint propagation of msg in INS. Figure 3 The taint propagation of the included instruction sequence and its operands.

[0053] Table 1

[0054]

[0055] By traversing the 10 execution information in INS as shown in Table 1, the byte position corresponding to the polluted operand is extracted from each instruction, and the byte sequence corresponding to the 10 instructions is obtained {(b0),(b0),(b1),(b1),(b 21 ),(b 22 ),(b 22 ),(b 21 )(b 22 ),(b 21 ,b 22 ),(b 21 ,b 22 )}.

[0056] Merge the consecutive byte offset positions in each instruction as candidate fields, and obtain the candidate fields {f0,f1,f 21,22}, and then add them to the set L.

[0057] It should be noted that Table 1 and Figure 3 It contains part of the execution information of msg on the server side. The rest of the execution information and the candidate field extraction method in the instruction can be deduced in the same way, which will not be repeated here. By traversing the complete execution information set INS, we can get the preliminary field division result set L = {f2,f0,f1,f3,f 4,5 ,f 6,7 ,f 8,9 ,f 21,22 ,f 10 ,f11 ,f 12 ,f 13 ,f 14 ,f 15 ,f 16 ,f 17 ,f 18 ,f 19 ,f 20}.

[0058] 3) Based on the binary instruction execution information set INS for processing msg and the preliminary field partition result set L of msg, sort the elements in L according to the position of the candidate field offset from small to large and update them to obtain L = {f0,f1,f2,f3,f 4,5 ,f 6,7 ,f 8,9 ,f 10 ,f 11 ,f 12 ,f 13 ,f 14 ,f 15 ,f 16 ,f 17 ,f 18 ,f 19 ,f 20 ,f 21,22}.

[0059] 4) Traverse the candidate fields L with adjacent field offsets in L i and L i+1 , that is, f (i,j) and f (j+1,k) , and judge f (i,j) and f (j+1,k) Are the instruction sequences of L similar in semantics? If so, i and L i+1 The corresponding two candidate fields f (i,j) and f (j+1,k) and merge them into a new candidate field f (i,k) .

[0060] Specifically, by traversing INS, we extract the operations f (i,j) and f (j+1,k) The instructions are stored in sets I and J. Secondly, the operator operator of each instruction inst is used as the instruction semantic feature to extract the operator sequence I composed of the operators of all instructions in instruction sets I and J. op and J op Then, the NW algorithm, namely the Needleman-Wunsch algorithm, is used to calculate the two operator sequences I op and J op The semantic similarity of is defined as follows:

[0061]

[0062] in,

[0063]

[0064] The parameter GAP_SCORE is the gap penalty, with a value of -2; MA_SCORE is the match score, with a value of 1; MISMA_SCORE is the mismatch score, with a value of -1, and the similarity threshold is 0.8.

[0065] For example, by analyzing the execution information of the adjacent candidate fields f0 and f1 as shown in Table 1, the operator sequence of f0 and f1 as shown in Table 2 can be obtained, that is, op and J op .

[0066] Table 2

[0067]

[0068] Using the NW algorithm to calculate I op ={movzx,cmp} and J op = The semantic similarity score of {movzx, cmp} is 1.0, which is greater than the threshold of 0.8. Therefore, the two candidate fields are merged into a new candidate field f 0,1 .

[0069] For example, by analyzing the adjacent candidate fields f shown in Table 1 20 and f 21,22 The execution information of f shown in Table 2 can be obtained. 20 and f 21,22 The operator sequence of I op and J op .

[0070] Table 3

[0071]

[0072]

[0073] Using the NW algorithm to calculate I op ={cmp,movzx,xor,mov,movzx} and J op = The semantic similarity score of {movzx, cmp} is -1.2, which is less than the threshold value of 0.8. Therefore, f 20 and f 21,22 The semantics are not similar and no need to merge.

[0074] The remaining execution information and adjacent candidate fields may be combined in the same manner and will not be described in detail here.

[0075] By using the instruction sequence semantic similarity analysis method as described above, two candidate fields f with similar semantics in the instruction sequence are merged. (i,j) and f (j+1,k) is a new candidate field f (i,k) , update L i+1 f (i,k) And L i Assign a value of Null. Finally, delete all Null elements in L to get the final field division result of msg.

[0076] for Figure 2 The final field division result is F = {f 0,1 ,f2,f3,f 4,5 ,f 6,7 ,f 8,9 ,f 10,11,12,13,14,15,16,17,18,19,20 ,f 21,22}.

[0077] Finally, it should be noted that the above description of the disclosed embodiments enables one skilled in the art to make or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for dividing protocol message fields based on the semantic similarity of binary instruction sequences, characterized in that: The following steps are involved: Step 1: Send the message msg to be divided into fields to the server with the function of recording instruction execution information, track the taint propagation in bytes, and extract the binary instruction execution information set INS for processing msg from the server's log information; Step 2: Based on the binary instruction execution information set INS of msg, according to the usage characteristics of the operands in the instructions, the polluted byte sequence in each instruction is extracted as the candidate field, and the preliminary field division result set L of msg is obtained; Step 3: Based on the binary instruction execution information set INS for processing msg and the preliminary field partition result set L of msg, sort the elements in L according to the offset position of the candidate fields from small to large and update L; Step 4: Extract the instruction sequence related to each candidate field in L from INS, use the NW algorithm to analyze the semantic similarity of the instruction sequence, merge the adjacent candidate fields with similar semantics in L, and output the final field division result of msg.

2. The method for dividing protocol message fields according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step A1: traverse INS. If INS has not been traversed yet, read an untraversed instruction inst and the operand set OPRD involved in inst from INS, and continue to execute step A2; otherwise, the preliminary field division of msg based on the analysis within the instruction is completed, and the preliminary field division result set L is output; Step A2: traverse OPRD. If OPRD has not been traversed yet, read an operand that has not been traversed from OPRD and continue to execute step A3. Otherwise, return to step A1; Step A3: Determine whether operand is contaminated. If so, extract the continuous byte offset position in msg of the contaminated operand from the execution information of inst, and add the continuous byte offset position as a candidate field to the set L; otherwise, return to step A2.

3. The method for dividing protocol message fields according to claim 1, characterized in that: The step 4 specifically includes the following steps: Step B1: Traverse L in order. If L has not been traversed yet, read the next set of candidate fields L that have not been traversed from L. i and L i+1 , proceed to step B2; Otherwise, execute step B3; Step B2: Determine L i and L i+1 Are the instruction sequences of L similar in semantics? If so, i and L i+1 Merge into a new candidate field and update L i+1 And L i Assign the value to Null; if L i and L i+1 If the instruction sequences are not semantically similar, return to B1; Step B3: Delete all null values ​​in L to obtain the final field division result of msg.

4. The method for dividing protocol message segments according to claim 1, characterized in that: The candidate field L in step B2 is judged i and L i+1 Whether the semantics of the instruction sequences are similar, specifically including the following steps: Step C1: Extract candidate fields L separately i and L i+1 The corresponding byte sequence bs (i,j) and bs (j+1,k) ; Step C2: traverse INS. If INS has not been traversed yet, take out an instruction inst that has not been traversed yet and a byte sequence Seq consisting of the polluted bytes accessed by inst from INS, and continue to execute C3; otherwise, execute step C4; Step C3: Determine whether Seq is bs (i,j) and bs (j+1,k) If Seq is bs (i,j) subsequence, then add inst to the candidate field L i Related instruction set I; if Seq is bs (j+1,k) subsequence, then add inst to the candidate field L i+1 Related instruction set J; if Seq is neither bs (i,j) A subsequence of (j+1,k) If the subsequence is , return to step C2; Step C4: Calculate the semantic similarity score of the instructions between set I and set J. If the semantic similarity score is greater than or equal to the similarity threshold, the candidate field L is determined. i and L i+1 For semantic similarity; Otherwise, determine the candidate field L i and L i+1 The semantics are not similar.

Images