An adaptive identification method for bit-level unknown protocol fields

Abstract

The application discloses a kind of self-adapting identification methods for bit level unknown protocol field, comprising: based on the PRE method of existing mature identification obvious byte level protocol field, and then extract target byte sample possibly containing bit level field;For the target byte sample possibly containing bit level field, identify fixed value field in advance;For target byte, according to different field division level, extract field pre-division strategy set;Global division model of bit level field is constructed, and the pareto solution set of each field pre-division strategy set is solved;Combining the pareto solution in all pareto solution sets, the utility value of each bit in target byte is calculated, which can be used as the basis for judging whether the corresponding bit is bit level field division point;Finally, using average utility value method, the corresponding bit is extracted as the final bit level field division point.The application is based on the global characteristics of protocol field, constructs global division model of bit level field, can effectively avoid the defect that the local mathematical characteristics of bit level field is fuzzy, significantly improves the recognition ability of existing PRE method to bit level field.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of industrial Internet of Things and information security, and specifically relates to a self-adaptive identification method for bit-level unknown protocol fields. BACKGROUND

[0002] Based on commercial and security considerations, many protocol developers do not disclose their protocol specification documents at present, especially some industrial equipment manufacturers. This leads to the existence of a large number of unknown non-standard protocol communications in the current network, which brings great challenges to some analysis techniques that need to rely on protocol specifications, such as intrusion detection, vulnerability mining, fuzz testing, protocol reuse, etc.

[0003] Protocol reverse engineering (PRE) can extract protocol syntax, semantics and synchronization information by monitoring and analyzing the network input / output, system behavior and instruction execution flow of the protocol entity without relying on the protocol description, so as to infer the format specification of the unknown protocol. Therefore, the PRE technology has been widely applied in the network security field such as intrusion detection, deep packet analysis, fuzz testing, etc. The analysis accuracy and precision directly affect the performance of these security analysis techniques based on PRE. Among them, the PRE method based on network traffic only needs to capture and analyze the network traffic, which is easy to implement and analyze, and has high universality, and has gradually become the mainstream PRE technology at present. However, the analysis precision of this kind of PRE technology is relatively poor, usually only reaching the fixed Byte level. Although some researchers have proposed PRE methods with higher precision, these methods are not only sensitive to the number and diversity of samples, but also can only achieve the fixed half-Byte level of field recognition precision. They still cannot completely solve the problem of adaptive identification of bit-level fields of different lengths.

[0004] In addition, most of the current network traffic-based PRE methods have high identification accuracy for fixed Byte-level or half-Byte-level fields, but they still cannot accurately and adaptively identify bit-level fields, and instead they will generally identify multiple bit-level fields as an independent Byte-level field. This not only leads to the incorrect identification of these bit-level fields and the inaccurate inference of their semantics, but also easily further leads to the low efficiency or even confusion of those security analysis techniques that rely on protocol specifications when dealing with these chaotic Byte-level fields, such as low protocol fuzz testing coverage, frequent protocol reuse error, etc. Therefore, how to further improve the identification ability of the existing PRE methods for these bit-level fields has become a research hotspot at present, especially for Bytes that may contain multiple bit-level fields of different lengths. SUMMARY

[0005] The application aims to provide an adaptive identification method for bit-level unknown protocol fields, which can effectively avoid the defect of bit-level field local mathematical feature ambiguity, significantly improve the identification ability of the existing PRE method for bit-level fields, and further provide strong support for other security analysis techniques that need to rely on protocol specifications.

[0006] The technical solution for achieving the application is: an adaptive identification method for bit-level unknown protocol fields, the specific steps of which are:

[0007] Identify byte-level protocol fields with obvious features based on the PRE method, and extract byte samples that may contain bit-level fields;

[0008] Identify fixed value fields for target byte samples that may contain bit-level fields;

[0009] For the target byte, generate a field pre-division strategy set according to different field division levels;

[0010] Construct a global bit-level field division model and solve the pareto solution set of each field pre-division strategy set;

[0011] Combine the pareto solutions in all pareto solution sets to calculate the utility value of each bit in the target byte, which can be used as a basis for determining whether the corresponding bit is a bit-level field division point;

[0012] Use the average utility value method to extract the corresponding bit as the final bit-level field division point.

[0013] Preferably, the target byte is divided into four field division levels of 2, 3, 4 and 5, and the field pre-division strategy set corresponding to each division level is extracted.

[0014] Preferably, for the target byte, the principle followed by generating the field pre-division strategy set according to different field division levels is:

[0015] Principle 1: Maintain the integrity of the fixed value field: always do not divide the fixed value field internally, that is, always ensure that the fixed field is a separate field;

[0016] Principle 2: Maintain the distinguishability of field features.

[0017] Preferably, the specific method for constructing a global bit-level field division model and solving the pareto solution set of each field pre-division strategy set is:

[0018] A global partition model of bit-level field is constructed: the "field internal determinacy" of the target byte is modeled as the total information entropy Sum_E(A m ), and the "field inter-non-correlation" of the target byte is modeled as the total mutual information Sum_I(A m ). As the two objective functions of the global partition model, they are specifically:

[0019]

[0020]

[0021] wherein, N represents the number of fields of the target byte after pre-partition, E(X) represents the information entropy, and I(X, Y) represents the mutual information;

[0022] The objective function value of each field pre-partition strategy A m in the field pre-partition strategy set S N is calculated, and the pareto sorting algorithm is adopted to sort all A m in each S N , to obtain multiple Pareto solution sets S * 2, S * 3, S * 4 and S * 5. Each S * N may contain one or more Pareto optimal solutions, each of which represents a better bit-level field partition strategy in the current solution space, and they are not dominated by each other.

[0023] Preferably, the information entropy E(X) of the field X is specifically:

[0024]

[0025] wherein, x i represents the i-th value of the field X, and p(x i ) represents the probability of x i in all values of the field X.

[0026] Preferably, the mutual information I(X, Y) of the field X and the field Y is specifically:

[0027]

[0028] wherein, y i represents the i-th value of the field Y, p(y j ) represents the probability of y j in all values of the field Y, and p(x i , y j) represents x i and y j Probability of simultaneous occurrence.

[0029] Preferably, the utility value V of each bit in the target byte is calculated by a simple weighting method in combination with the pareto solution in the pareto solution set i The calculation formula is as follows:

[0030]

[0031] Where, Num N,i is the frequency of the i-th bit as a field division point, Num N is the total division frequency of all bits, w N is the weight corresponding to different division levels.

[0032] Compared with the prior art, the present application has the following advantages:

[0033] (1) High precision: the global division model of bit-level fields is constructed by using the global characteristics of "field internal determinacy and field non-correlation", which can effectively avoid the defects of bit-level field local mathematical characteristics ambiguity and significantly improve the recognition ability of the existing PRE method for bit-level fields, so that the field division precision reaches the bit level.

[0034] (2) High accuracy: the optimal field division strategy extracted according to different field division levels can accurately cover the target protocol specification, and the finally extracted global optimal field division strategy has high accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is the basic flowchart of the adaptive recognition method for bit-level unknown protocol fields of the present application. DETAILED DESCRIPTION

[0036] The present application will be further described in detail below in combination with the drawings and implementation examples. The following examples are implemented on the premise of the technical scheme of the present application, and detailed implementation methods and processes are given, but the protection scope of the present application is not limited to the following examples.

[0037] The present application realizes the adaptive and accurate extraction of bit-level fields by modeling the global characteristics of bit-level fields. On the one hand, the "bit-level field division" problem is modeled as a "multi-objective decision" model, which can effectively avoid the problem of bit-level field local mathematical characteristics ambiguity; on the other hand, the "multi-objective decision" model is solved, and the utility value of each bit is further calculated, realizing the adaptive and accurate extraction of bit-level fields.

[0038] The specific implementation steps of the present application are shown in steps 101 to 106, and the specific steps are:

[0039] Step 101: Based on the existing mature PRE method, the byte-level protocol field with obvious characteristics is identified, and then the target byte sample that may contain bit-level fields is extracted;

[0040] Step 102: For the target byte sample that may contain bit-level fields, the fixed value field is identified in advance;

[0041] By comparing each corresponding bit value in all samples of the target byte, the fixed value bit is found to determine the fixed value field;

[0042] Step 103: For the target byte, generate a field pre-division strategy set according to different field division levels;

[0043] Based on the fixed value field, for the target byte, generate the corresponding field pre-division strategy set S N according to different field division levels L N . This step needs to follow two principles:

[0044] Principle 1: Maintain the integrity of the fixed value field. Always do not divide the fixed value field internally, that is, always ensure that the fixed field is a separate field;

[0045] Principle 2: Maintain the distinguishability of field characteristics. L N The higher, the lower the field diversity, the less obvious the mathematical characteristics, and thus it is difficult to serve as a basis for field division. Therefore, maintaining the distinguishability of field characteristics can be a reliable basis for subsequent field division.

[0046] Therefore, the present application only presets the pre-division level L N of the target byte as L2, L3, L4, and L5, which corresponds to dividing the target byte into 2, 3, 4, and 5 fields, respectively. Each L N corresponds to a field pre-division strategy set S N ={A0,A1,…,A m ,…,A M-1}, where A m =[0,a0,a1,…,a n ,…,a N-2 ](0<a n <a n+1 <8) represents the mth division strategy in S N .

[0047] Step 104: Construct a global bit-level field division model and solve the pareto solution set of each field pre-division strategy set;

[0048] Build the bit-level field global partition model: the "field internal determinacy" of the target byte is modeled as the total information entropy Sum_E(A m ), that is, formula (7), and the "field inter-non-correlation" of the target byte is modeled as the total mutual information Sum_I(A m ), that is, formula (8), and then as two objective functions of the global partition model.

[0049]

[0050]

[0051] In the formula, x i represents the i-th value of field X, and p(x i ) represents the probability of x i appearing in all values of field X.

[0052] The information entropy E(X) of field X is specifically:

[0053]

[0054] In the formula, x i represents the i-th value of field X, and p(x i ) represents the probability of x i appearing in all values of field X. The mutual information I(X,Y) of field X and field Y is specifically:

[0055]

[0056] In the formula, y i represents the i-th value of field Y, p(y j ) represents the probability of y j appearing in all values of field Y, and p(x i ,y j ) represents the probability of x i and y j appearing at the same time.

[0057] The objective function value of each field pre-partition strategy A N in S m is calculated, and the pareto sorting algorithm is used to sort all A N in each S m . Obtain multiple Pareto solution sets S * 2, S * 3, S * 4 and S * 5, and each S * NThere may be one or more Pareto optimal solutions in , each of which represents a better bit-level field partitioning strategy in the current solution space, and they do not dominate each other.

[0058] Step 105: Calculate the utility value of each bit in the target byte by combining all the Pareto solutions in the Pareto solution set. The utility value can be used as a basis for determining whether the corresponding bit is a bit-level field division point.

[0059] Based on all S * N The present invention adopts the "Simple Additive Weighting (SAW) algorithm" to calculate the utility value V i , which can be used as a basis for judging whether the i-th bit can be used as a field division position. The calculation formula is as follows:

[0060]

[0061] Among them, Num N,i is the frequency of the i-th bit as the field division point, Num N is the total division frequency of all bits, w N is the weight corresponding to different classification levels. In this invention, the weight w N The settings are as follows:

[0062] Table 1 Weight w N set up

[0063]

[0064] Step 106: Use the average utility value method to extract the corresponding bit as the final bit-level field division point.

[0065] V i Sort in descending order, the first N bits will be used as L N The field division point and constitute each L N The optimal field partitioning strategy A_opt * N In addition, the present invention adopts the “average utility value method” to convert V i The bit with a larger utility value than the average value is used as the global optimal field partitioning strategy A_opt * glo The field split point in .

[0066] In conclusion, the application models the global characteristics of the protocol field, can effectively avoid the defects of bit-level field local mathematical characteristics ambiguity, and can realize adaptive and accurate identification of the bit-level field, and effectively improve the identification ability of the existing PRE technology on the bit-level field.

[0067] The above merely describes the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

[0068] It should be understood that, in order to simplify the present application and help those skilled in the art to understand various aspects of the present application, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes described in a single embodiment, or with reference to a single figure. However, the present application should not be interpreted as the features included in the exemplary embodiments are all necessary technical features of the patent claims of the present application.

[0069] It should be understood that the modules, units, components, etc. included in the device of one embodiment of the present application can be adaptively changed to be arranged in a device different from the embodiment. Different modules, units or components included in the device of the embodiment can be combined into one module, unit or component, or divided into multiple sub-modules, sub-units or sub-components.

Claims

1. An adaptive recognition method for bit-level unknown protocol fields, characterized in that: The specific steps are: Based on the PRE method, we identify byte-level protocol fields with obvious characteristics and extract byte samples that may contain bit-level fields. For target byte samples that may contain bit-level fields, identify fixed-value fields; For the target byte, classify the levels according to different fields and generate a set of field pre-partition strategies; Construct a global partitioning model for bit-level fields and solve the Pareto solution set of each field pre-partitioning strategy set. The specific method is as follows: Construct a global partitioning model for bit-level fields: Model the internal determinism of the target byte field as the total information entropy Sum_E(A m ), the non-correlation between the fields of the target byte is modeled as the total mutual information Sum_I(A m ), as the two objective functions of the global partitioning model, are specifically: Where N represents the number of fields after the target byte is pre-divided, E(X) represents information entropy, and I(X,Y) represents mutual information. The information entropy E(X) of field X is specifically: Where x i represents the i-th value of field X, p(x i ) represents x i The probability of appearing in all values ​​of field X; The mutual information I(X,Y) between field X and field Y is specifically: Where y i Indicates the i-th value of field Y, p(y j ) represents y j The probability of appearing in all values ​​of field Y, p(x i ,y j ) represents x i and y j Probability of simultaneous occurrence; Calculate the field pre-partitioning strategy set S N Each field in the pre-partition strategy A m The objective function value is used, and the Pareto sorting algorithm is used to sort each S N All A's m Sort and obtain multiple Pareto solution sets S * 2. S * 3. S * 4 and S * 5, and each S * N There may be one or more Pareto optimal solutions, each of which represents a better bit-level field partitioning strategy in the current solution space, and they do not dominate each other; Combining all the Pareto solutions in the Pareto solution set, the utility value of each bit in the target byte is calculated. This utility value can be used as a basis for determining whether the corresponding bit is a bit-level field division point. Calculate the utility value V of each bit in the target byte by a simple weighted method i , the calculation formula is as follows: Among them, Num N,i is the frequency of the i-th bit as the field division point, Num N is the total division frequency of all bits, w N is the weight corresponding to different classification levels; The average utility value method is used to extract the corresponding bit as the final bit-level field division point.

2. The adaptive identification method for bit-level unknown protocol fields according to claim 1, characterized in that: The target bytes are divided into four field division levels: 2, 3, 4 and 5, and the field pre-division strategy set corresponding to each division level is extracted.

3. The adaptive identification method for bit-level unknown protocol fields according to claim 1, characterized in that: For the target byte, the principles for generating a set of field pre-partitioning strategies are as follows: Principle 1: Maintain the integrity of fixed-value fields: Never perform internal divisions on fixed-value fields, that is, always ensure that fixed fields are a separate field; Principle 2: Keep field features distinguishable.

Images