A black-box fuzzing method and device based on feedback enhancement

Abstract

The application discloses a black box fuzzing method and device based on feedback enhancement. The method divides the field boundary by the structural feature change of the message itself, and constructs an automatic semantic inference method on the basis, learns the abstract connection between the field features and semantic categories from the public protocol, and applies the learned knowledge to infer the semantics of each field in the private protocol. Finally, the feedback message of the device is collected in the fuzzing process to establish a new feedback mechanism, and the variation type and variation strategy are adjusted according to the feedback information. The application can automatically identify the field boundary and semantic information based on the network communication data of the industrial field, without manual participation, and take it as the basis for establishing the feedback mechanism, and has good test effect on the complex and private industrial control protocol.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial control system security, and particularly relates to a black-box fuzzing test method and device based on feedback enhancement. BACKGROUND

[0002] Industrial Control Systems are process control systems that use various automation devices to collect and monitor real-time data of the entire industrial production process and make timely adjustments. It includes data monitoring and collection systems, distributed control systems, programmable logic controllers, and other systems. Industrial control systems are widely used in key industries and fields such as chemical industry and power industry, and are the backbone of the foundation industry on which the entire country and society depend. With the rapid development of Internet technology in recent years and the key breakthroughs in hardware technology, the trend of Internet of Things has become increasingly unstoppable. The productivity improvement and huge economic benefits brought about by it have led to an informationization and networkization trend in various industries, and the industrial control field is no exception. However, as more and more traditional industrial control devices are connected to the industrial Internet, many security issues have also arisen. Since most industrial control devices were designed with only intranet communication in mind, they lack effective and strict identity verification, and the operating systems used lack timely updates and vulnerability fixes. Once these industrial control devices are attacked, they may cause data leakage and industrial production stagnation, or even further threaten the safety of the entire industrial system and the workers. How to quickly and effectively exploit industrial control system vulnerabilities has become an important issue.

[0003] Protocol fuzzing is a mainstream method for vulnerability exploitation today. It generates a large number of random, abnormal, or boundary data to simulate various possible attack scenarios, sends data to the target protocol implementation, and monitors the response. If the protocol crashes or behaves abnormally when processing these data, the testing tool will save the test cases and analyze the location and cause of the abnormality to determine possible vulnerabilities in the device.

[0004] Traditional protocol fuzzing methods rely on prior knowledge and need to construct seeds for fuzzing based on known syntax structures. However, in industrial scenarios, most devices have a private communication standard, and the internal execution information is often difficult to obtain, so we cannot adjust the mutation strategy of fuzzing based on feedback information. Therefore, fuzzing methods for unknown protocols are needed to study the security of industrial control devices. SUMMARY

[0005] The present application aims to address the shortcomings and deficiencies of current private protocol fuzzing methods and provides a black-box fuzzing test method and device based on feedback enhancement.

[0006] The purpose of the present application is achieved by the following technical solutions:

[0007] In a first aspect, the present application provides a black-box fuzzing method based on feedback enhancement, which comprises the following steps:

[0008] Collecting network communication traffic of the industrial control system, analyzing the change distribution of the internal value of each message using the message structure characteristics, and dividing the field boundaries according to the internal structural differences between the fields;

[0009] Building an automated semantic inference method to learn the abstract relationship between field characteristics and semantic categories from the public protocol, and applying the learned knowledge to infer the semantics of each field in the private protocol;

[0010] Based on the results of field division and semantic extraction, a fuzzing method based on feedback enhancement is constructed, the device feedback messages are collected during the fuzzing process to establish a new feedback mechanism, and the weight of the variation strategy and the variation type is adjusted according to the feedback information.

[0011] Further, the message structure characteristics are the similarity of consecutive bytes in the message, that is, the similarity of adjacent bytes is obtained by summing and averaging the same or of each bit of the adjacent two bytes; then the structural feature change of the adjacent bytes is calculated; finally, according to the message structure characteristic change curve, the curve is smoothed using a Gaussian filter, and the inflection point of the rising period of the curve is taken as the field boundary.

[0012] Further, the semantic inference method comprises the following three parts: field feature extraction, fine-grained semantic aggregation, and semantic inference based on similarity.

[0013] Further, the field feature integrates the field itself feature and the context feature of the field, and the context feature of the field represents the timing relationship between the field and the field at the same byte offset position in the message sequence and the relationship between the field and the adjacent field.

[0014] Further, the field feature extraction is based on three kinds of gated recurrent network units, the first kind of gated recurrent network unit G i is used to extract the field itself feature, the second kind of gated recurrent network unit G o is used to extract the timing relationship between the field and the field at the same byte offset position in the message sequence, and the third kind of gated recurrent network unit G r is used to extract the relationship between the field and the adjacent field, and finally the multiple gated recurrent network units are integrated into a large neural network model.

[0015] Further, the field feature extraction is specifically:

[0016] First, each byte of the sth field of the fth message is used as G i The input and output hidden state That is, the extracted features of the field itself; taking the longest length of each field as the benchmark, fill in the fields that are less than the length;

[0017] Secondly, G i The extracted features of each field are input into G o In, G o The hidden state of the output is recorded as For a message sequence consisting of N fields, the partial features of all fields form a sequence

[0018] Finally, using bidirectional GRU, G r , processing sequence use The corresponding output is the final representation of the field features, and the features of each field are recorded as r s , the field feature R of the entire message sequence is expressed as (r 1 ,r 2 ,…,r N ).

[0019] Furthermore, the semantic aggregation is implemented based on text similarity. For known protocols, a natural language model is used to convert semantic descriptions into semantic vectors, and clustering is performed based on the similarity between the vectors.

[0020] Furthermore, the similarity-based semantic inference includes:

[0021] During the model training phase, for fields belonging to the same semantic category, the feature similarity between them needs to be higher, and vice versa. The loss function L established based on this feature is as follows:

[0022]

[0023] Where P represents the positive sample set, A represents the negative sample set, sim represents the inner product of two vectors, and τ represents the temperature parameter;

[0024] In the model utilization stage, after extracting the field features of the unknown protocol, the similarity between each field feature and the center of each semantic category is calculated, and the semantics of the field is inferred based on the similarity.

[0025] Furthermore, the collection device feedback message is used to establish a new feedback mechanism, specifically:

[0026] A complete deterministic mutation is performed on the seeds in the current seed library which have not been subjected to deterministic mutation, and the seeds are sent to the device under test, and the feedback messages with high similarity are merged into the same category; if different response types are triggered, the test case is regarded as a new unique seed and added to the seed library; otherwise, the test case is not saved;

[0027] When there is no seed in the seed library which has not been subjected to deterministic mutation, a seed is randomly selected for a havoc mutation, and if the number of mutations reaches the preset number without triggering a new response type, the fuzz testing is terminated;

[0028] Meanwhile, in the process of deterministic mutation, the device state of each round of testing is collected, different states are given different weights, and then the weight sum corresponding to each mutation type is calculated, and the next round of testing will dynamically adjust the probability of occurrence of each mutation type in the next round of testing according to the proportion of the weight sum of each mutation type.

[0029] In the second aspect, the present application provides a black box fuzz testing device based on feedback enhancement, comprising the following modules:

[0030] The field division module: the internal value change distribution of each message is analyzed by using the structure characteristics of the message itself, and the field boundaries are divided according to the internal structural difference between the fields;

[0031] The semantic inference module: an automated semantic inference method is constructed to learn the abstract relationship between the field characteristics and the semantic categories from the public protocol, and the learned knowledge is applied to infer the semantics of each field in the private protocol;

[0032] The fuzz testing module: the device feedback messages are collected in the fuzz testing process to establish a new feedback mechanism, and the mutation strategy and the weight of the mutation type are adjusted according to the feedback information.

[0033] The present application has the beneficial effects that the black box fuzz testing method based on feedback enhancement proposed by the present application automatically identifies the field boundaries and semantic information based on the network communication data of the industrial field, and uses the same as the basis for establishing the feedback mechanism, adjusts the mutation type and mutation strategy of the fuzz testing according to the real-time feedback, and has good test effect on complex and private industrial control protocols. The method of the present application is a general black box fuzz testing method, which is not affected by the type of protocol, does not require too much prior knowledge, is practical and efficient. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below only constitute some embodiments of the present application. Based on these drawings, other drawings can be obtained by one of ordinary skill in the art without any creative effort.

[0035] Figure 1 is an example of data preprocessing in the method of the present application;

[0036] Figure 2 is a flowchart of the automated semantic inference method in the method of the present application;

[0037] Figure 3 is a flowchart of field feature extraction in the method of the present application. DETAILED DESCRIPTION

[0038] In order to better understand the technical solutions of the present application, the embodiments of the present application will be described in detail below with reference to the drawings.

[0039] It should be clear that the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by one of ordinary skill in the art without any creative effort are within the scope of protection of the present application.

[0040] The terms used in the embodiments of the present application are only for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0041] The present embodiment provides a black box fuzzing method based on feedback enhancement, which comprises the following steps:

[0042] (1) Collect the network communication traffic of the industrial control system, each traffic packet contains a plurality of message sequences.

[0043] Using the message structure characteristics, the change distribution of the internal value of each message is analyzed, and the boundaries of the fields are divided according to the internal structural difference between the fields. The message structure characteristics are the similarity of the consecutive bytes in the message, and the calculation method is as follows:

[0044]

[0045] Wherein, P and Q represent two adjacent bytes, i represents the i-th bit in the byte, represents the same or operation, and D represents the similarity. Further, the structural characteristic change of adjacent bytes can be represented by the following formula:

[0046] △D k = (D(m k , m k+1 ) - D(m k-1 , m k ))

[0047] where k represents the kth position in the message m, m k represents the kth byte in the message m, and k ∈ (0, n) for a message of length n.

[0048] Thus, according to the message structure feature variation curve, the inflection point of the rising period of the curve is taken as the boundary of the field after the curve is smoothed using a Gaussian filter.

[0049] (2) Data preprocessing is required before semantic inference. As shown in Figure 1 , an open-source protocol parser such as wireshark or tshark is used to parse the traffic packet, and the result is output in json format. Then, the protocol field name and description information are obtained from the json file and stored in csv format.

[0050] (3) An automated semantic inference method is constructed to learn the abstract relationship between field features and semantic categories from public protocols (i.e., known protocols) and apply the learned knowledge to infer the semantics of each field in private protocols (i.e., unknown protocols).

[0051] Specifically, as shown in Figure 2 , the method includes three parts, the first of which is field feature extraction, the second of which is fine-grained semantic aggregation, and the third of which is similarity-based semantic inference. The entire process can be divided into two stages. In the model training stage, for fields belonging to the same semantic category, the feature similarity between them needs to be higher, and vice versa. In the model utilization stage, after the field feature extraction method is used to extract the field features of the unknown protocol, the similarity between each field feature and the center of each semantic category is calculated, and the semantic to which the field belongs is inferred according to the similarity.

[0052] The field feature extraction method is based on three kinds of gated recurrent unit (GRU) units. The first kind of gated recurrent unit G i is used to extract the features of the field itself, the second kind of gated recurrent unit G o is used to extract the timing relationship between the field and the fields at the same byte offset position in the message sequence, and the third kind of gated recurrent unit G r is used to extract the relationship between the field and the adjacent fields. Finally, the multiple GRU units are integrated into a large neural network model.

[0053] Specifically, as shown in Figure 3As shown, each byte of the sth field of the fth message is used as G i The input and output hidden state This is the feature of the field itself that is extracted. Since not every message has the same length for this field, the longest length of the field is used as a benchmark, and the fields that are less than the length are padded with 256, such as Figure 3 The "padding" in the , until all fields at that position reach the same length;

[0054] Secondly, G i The extracted features of each field are input into G o The more messages a message sequence contains, the more G o The more input, the more G o The hidden state of the output is recorded as For a message sequence consisting of N fields, the partial features of all fields form a sequence

[0055] Finally, using bidirectional GRU, G r , process the sequence use The corresponding output is the final representation of the field features, and the features of each field are recorded as r s , the field feature R of the entire message sequence can be expressed as (r 1 ,r 2 ,…,r N ).

[0056] Fine-grained semantic aggregation is to aggregate multiple similar fine-grained semantics into coarse-grained semantic categories based on semantic similarity. This step aims to improve the scalability of the method. For some proprietary semantics appearing in private protocols, the meaning of the field can be roughly inferred by calculating its similarity with existing public semantics, rather than matching one by one in a few pattern rules reflected in semantics such as length fields, serial number fields, and function code fields. The specific implementation of semantic aggregation is based on text similarity. For known protocols, open source protocol decoders such as wireshark or tshark are used to obtain the semantic description of each field. The semantic description is converted into a semantic vector using a natural language model, and clustering is performed based on the similarity between the vectors.

[0057] During the model training phase, for fields belonging to the same semantic category, the feature similarity between them needs to be higher, and vice versa. The loss function L established based on this feature is as follows:

[0058]

[0059] where P represents the set of all positive samples, A represents the set of negative samples, sim represents the inner product of two vectors, and τ represents a temperature parameter.

[0060] (4) Based on the results of field division and semantic extraction, a fuzzy testing method based on feedback enhancement is constructed, device feedback messages are collected in the fuzzy testing process to establish a new feedback mechanism, and the weights of variation strategies and variation types are adjusted according to the feedback information; specifically:

[0061] The change of feedback message type reflects the change of the execution path of the device under test to a certain extent. The first step of fuzzy testing is to perform a complete deterministic variation on the seeds in the current seed library that have not been subjected to deterministic variation, including but not limited to bit flipping, bit addition and subtraction, special value replacement, etc., and send them to the device under test. The Smith-Waterman algorithm is used to calculate the similarity of the feedback message and the message in the response library, and the hierarchical clustering method is used to merge the feedback messages with high similarity to the same class. If different response types are triggered, the test case is considered as a new unique seed and added to the seed library; otherwise, the test case will not be saved.

[0062] When there is no seed in the seed library that has not been subjected to deterministic variation, a seed is randomly selected for havoc variation. If the number of variations reaches a certain number without triggering a new response type, the fuzzy testing is terminated.

[0063] At the same time, in the process of deterministic variation, the device state of each round of testing is collected, different weights are assigned to different states, and the weight sum corresponding to each variation type is calculated. The probability of occurrence of each variation type in the next round of testing will be dynamically adjusted according to the proportion of the weight sum of each variation type, and each 500 test cases are recorded as 1 round in this embodiment. In the first round of testing, the mutation probability of each variation strategy is equal, and the dynamic adjustment strategy of each round thereafter is as follows:

[0064] ① Define the weight of each connection state, denoted by S, define the connection timeout as 7, the successful response and trigger a new path as 4, and the successful response but not trigger a new path as 1;

[0065] ② For variation type K, the weight sum of the variation type is calculated as where h is the number of all states triggered by the variation type, S represents the weight of the connection state, and count is the number of connection states;

[0066] ③ Based on the weight sum of the variation type calculated in step ②, the mutation probability of the variation type is adjusted to where l represents the total number of variation types.

[0067] In another aspect, the present application also provides a black-box fuzz testing device based on feedback enhancement, which comprises:

[0068] The field division module: the variation distribution of the internal value of each message is analyzed by using the structural characteristics of the message itself, and the field boundary is divided according to the internal structural difference between the fields.

[0069] The semantic inference module: an automated semantic inference method is constructed, the abstract relationship between the field characteristics and the semantic categories is learned from the public protocol, and the learned knowledge is applied to infer the semantics of each field in the private protocol.

[0070] The fuzz testing module: the device feedback messages are collected in the fuzz testing process to establish a new feedback mechanism, and the weight of the variation strategy and the variation type is adjusted according to the feedback information.

[0071] The implementation of each module can refer to the steps in the black-box fuzz testing method based on feedback enhancement described above.

[0072] Corresponding to the above-mentioned embodiment of the black-box fuzz testing method based on feedback enhancement, the present application also provides an embodiment of a black-box fuzz testing device based on feedback enhancement. The black-box fuzz testing device based on feedback enhancement provided by the embodiment of the present application comprises a memory and one or more processors, the memory stores executable code, and the processor executes the executable code to implement the black-box fuzz testing method based on feedback enhancement in the above-mentioned embodiment.

[0073] The embodiment of the black-box fuzz testing device based on feedback enhancement provided by the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or realized by hardware or a combination of software and hardware.

[0074] The embodiment of the present application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the black-box fuzz testing method based on feedback enhancement in the above-mentioned embodiment.

[0075] The computer readable storage medium can be an internal storage unit of any of the aforementioned devices with data processing capability, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any of the aforementioned devices with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both an internal storage unit and an external storage device of any of the aforementioned devices with data processing capability. The computer readable storage medium is used to store the computer program and other programs and data required by the aforementioned devices with data processing capability, and can also be used to temporarily store data that has been output or is to be output.

[0076] The above embodiments are used to explain and illustrate the present application, but are not intended to limit the present application. Any modifications and changes made to the present application within the spirit and protection scope of the claims fall within the protection scope of the present application.

Claims

1. A black box fuzz testing method based on feedback enhancement, characterized in that: include: Collect network communication traffic from industrial control systems, analyze the distribution of internal values ​​within each message using the message's structural characteristics, and delineate field boundaries based on the inherent structural differences between fields. Build an automated semantic inference method to learn the abstract relationship between field features and semantic categories from public protocols, and apply the learned knowledge to infer the semantics of each field in private protocols; Based on the results of field segmentation and semantic extraction, a feedback-enhanced fuzz testing method is constructed. During the fuzz testing process, device feedback messages are collected to establish a new feedback mechanism. The mutation strategy and the weight of the mutation type are adjusted according to the feedback information. The method of collecting device feedback messages to establish a new feedback mechanism is specifically as follows: Perform a complete deterministic mutation on seeds in the current seed library that have not yet undergone deterministic mutation and send them to the device under test. Combine feedback messages with high similarity into the same category. If a different response type is triggered, the test case is treated as a new unique seed and added to the seed library. Otherwise, the test case is not saved. When there are no seeds in the seed library that have not undergone deterministic mutation, a seed is randomly selected to undergo a havoc mutation. If the number of mutations reaches the preset number and no new response type is triggered, the fuzz test is terminated. At the same time, during the deterministic mutation process, the device status of each round of testing is collected, different weights are assigned to different statuses, and then the sum of the weights corresponding to each mutation type is calculated. The next round of testing will dynamically adjust the probability of each mutation type occurring in the next round of testing based on the proportion of the sum of the weights of each mutation type.

2. The black box fuzz testing method based on feedback enhancement according to claim 1 is characterized in that The structural feature of the message itself is the similarity of consecutive bytes in the message, that is, the similarity of adjacent bytes is obtained by adding and averaging each bit of two adjacent bytes; then the change in the structural features of adjacent bytes is calculated; finally, based on the change curve of the structural features of the message itself, a Gaussian filter is used to smooth the curve, and the inflection point of the rising period of the curve is used as the field boundary.

3. The black box fuzz testing method based on feedback enhancement according to claim 1 is characterized in that The semantic inference method includes the following three parts: field feature extraction, fine-grained semantic aggregation, and similarity-based semantic inference.

4. The black box fuzz testing method based on feedback enhancement according to claim 1 is characterized in that The field characteristics combine the characteristics of the field itself and the context characteristics of the field. The context characteristics of the field represent the timing relationship between the field and the fields at the same byte offset position in the message sequence, as well as the relationship between the field and adjacent fields.

5. The black box fuzz testing method based on feedback enhancement according to claim 3 is characterized in that: The field feature extraction is based on three types of gated recurrent network units. The first type of gated recurrent network unit G i Used to extract the characteristics of the field itself, the second gated recurrent network unit G o The third gated recurrent network unit G is used to extract the temporal relationship between the field and the field at the same byte offset position in the message sequence. r It is used to extract the relationship between fields and adjacent fields, and finally integrates multiple gated recurrent network units into a large neural network model.

6. The black box fuzz testing method based on feedback enhancement according to claim 5 is characterized in that: The field feature extraction is specifically as follows: First, each byte of the sth field of the fth message is used as G i The input and output hidden state That is, the extracted features of the field itself; The longest length of each field is used as a benchmark, and the fields that are less than the length are filled; Secondly, G i The extracted features of each field are input into G o In, G o The hidden state of the output is recorded as For a message sequence consisting of N fields, the hidden states corresponding to all fields form a sequence Finally, using bidirectional GRU, G r , processing sequence use The corresponding output is the final representation of the field features, and the features of each field are recorded as r s , the field feature R of the entire message sequence is expressed as (r 1 ,r 2 ,…,r N ).

7. The black box fuzz testing method based on feedback enhancement according to claim 3 is characterized in that: The semantic aggregation is implemented based on text similarity. For known protocols, a natural language model is used to convert semantic descriptions into semantic vectors, and clustering is performed based on the similarity between the vectors.

8. The feedback-enhanced black box fuzz testing method according to claim 6, characterized in that: The similarity-based semantic inference includes: During the model training phase, for fields belonging to the same semantic category, the feature similarity between them needs to be higher, and vice versa. The loss function L established based on this feature is as follows: Where P represents the positive sample set, A represents the negative sample set, sim represents the inner product of two vectors, and τ represents the temperature parameter; In the model utilization stage, after extracting the field features of the unknown protocol, the similarity between each field feature and the center of each semantic category is calculated, and the semantics of the field is inferred based on the similarity.

9. A black box fuzz testing device based on feedback enhancement, characterized in that: include: Field division module: Utilizes the structural characteristics of the message itself to analyze the distribution of changes in the internal values ​​of each message and divides the field boundaries according to the inherent structural differences between the fields; Semantic Inference Module: Builds an automated semantic inference method to learn the abstract connections between field features and semantic categories from public protocols, and applies the learned knowledge to infer the semantics of each field in private protocols. Fuzz testing module: During the fuzz testing process, device feedback messages are collected to establish a new feedback mechanism, and the weights of mutation strategies and mutation types are adjusted based on the feedback information. The collection of device feedback messages to establish a new feedback mechanism is specifically as follows: Perform a complete deterministic mutation on seeds in the current seed library that have not yet undergone deterministic mutation and send them to the device under test. Combine feedback messages with high similarity into the same category. If a different response type is triggered, the test case is treated as a new unique seed and added to the seed library. Otherwise, the test case is not saved. When there are no seeds in the seed library that have not undergone deterministic mutation, a seed is randomly selected to undergo a havoc mutation. If the number of mutations reaches the preset number and no new response type is triggered, the fuzz test is terminated. At the same time, during the deterministic mutation process, the device status of each round of testing is collected, different weights are assigned to different statuses, and then the sum of the weights corresponding to each mutation type is calculated. The next round of testing will dynamically adjust the probability of each mutation type occurring in the next round of testing based on the proportion of the sum of the weights of each mutation type.

Images