Efficient grey-box fuzzing method and system for industrial control protocol

Abstract

The application discloses a kind of grey box fuzzy test method and system for industrial control protocol, method includes extracting field in industrial control protocol;Initialize shared memory and bitmap;According to the branch information in bitmap and heuristic field scheduling strategy, select the field of this round variation, and carry out variation, generate test sample;Start the system under test, send test sample to the system under test after plugging and test, and close the system under test after testing is completed;Whether the system under test collapses is monitored, if yes, record crash information;Otherwise, obtain branch coverage, judge whether new branch is generated in this test, if yes, after updating global bitmap, next step is carried out, otherwise, next step is directly entered;Whether iteration number / crash number meets preset condition, if yes, output coverage, crash information and its corresponding test sample, otherwise, update the variation number of variation field, and when new branch is generated, update branch coverage, then return field selection step.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to an efficient gray-box fuzz testing method and system for industrial control protocols. Background Technology

[0002] With the rapid development of information technology, industrial control systems (ICS) are widely used worldwide. According to the definition given by the National Institute of Standards and Technology (NIST), ICS is a general term encompassing various types of control systems. As an important means of communication among ICS components, the ICS protocol provides a framework for communication and control in industrial control systems and is a key factor in ensuring the efficiency and safety of industrial processes. However, the ICS protocol has various vulnerabilities in its implementation, making it a consistent target for hackers.

[0003] Fuzzing is one of the mainstream vulnerability discovery techniques, known for its ease of deployment, good scalability, and applicability. Based on how test cases are generated, fuzzing techniques are divided into two categories: mutation-based and generation-based. Mutation-based fuzzing requires no syntax guidance, generating input by randomly mutating a seed at the bit level; while generation-based fuzzing is suitable for scenarios with highly structured inputs, requiring the input to be constructed based on a certain syntax. Currently, both types of fuzzing tools are widely used and have uncovered numerous vulnerabilities. However, these tools do not fully consider the characteristics of ICS protocol software, leading to inefficiency, mainly in the following two aspects:

[0004] 1) The ICS protocol is highly structured and has strict syntactic constraints. Figure 1 The message structure of the Modbus TCP industrial control protocol is given, which has three fields: the MBAP header, the function code, and the data field, each with a specified size. If the constructed input does not meet its format requirements, it will be filtered directly during the format parsing stage of the ICS protocol software, preventing it from entering deeper protocol states.

[0005] 2) Different fields in the ICS protocol have varying degrees of importance. ICS protocol messages typically contain several fields, each with significantly different importance for coverage or vulnerability discovery. However, current ICS protocol fuzzing techniques treat every field in the protocol message equally, ignoring their differences. Summary of the Invention

[0006] To address the aforementioned shortcomings in existing technologies, the present invention provides an efficient gray-box fuzzy testing method and system for industrial control protocols, which solves the problem of low efficiency through coverage feedback guidance and protocol field selection strategies.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0008] Firstly, a gray-box fuzz testing method for industrial control protocols is provided, which includes the following steps:

[0009] S1. Extract different fields and state transition relationships from the industrial control protocol;

[0010] S2. Initialize shared memory and bitmap, wherein the bitmap includes a single bitmap representing program branch information triggered by this fuzz test and a global bitmap representing global branch information;

[0011] S3. Based on the branch information in the bitmap and the heuristic field scheduling strategy, select the field to be mutated in this round, use different mutation operators to mutate the selected field, and generate test cases.

[0012] S4. Start the system under test, send test samples to the system under test after the instrumentation is completed, and shut down the system under test after the test is completed.

[0013] S5. Monitor whether the system under test crashes during the test. If so, record the crash information and proceed to step S6. Otherwise, obtain the branch coverage of the system under test during the test and determine whether the current fuzz test generates a new branch. If so, update the global bitmap and proceed to step S6. Otherwise, proceed directly to step S6.

[0014] S6. Determine whether the number of iterations or the number of crashes meets the preset conditions. If yes, output the coverage, crash information and its corresponding test cases. Otherwise, update the number of mutations in the mutation field and update the branch coverage when a new branch is generated. Then return to step S3.

[0015] The beneficial effects of the above technical solution are as follows: This solution uses the instrumented system under test, which can statistically analyze the branch coverage during program execution and perform heuristic scheduling of the protocol at the field level based on the coverage feedback, giving fields with exploration potential more opportunities for mutation. This solution fully considers the characteristics of industrial control protocol message fields and effectively improves the quality of test samples.

[0016] This approach starts the system under test at the beginning of each fuzzing iteration and terminates the process at the end, ensuring noise-free fuzzing of industrial control protocols. This guarantees that each test case starts execution in the same initial server state, preventing previously sent messages from leading the server to a new state, exploring numerous additional branches, and causing incorrect evaluations of the protocol fields in the current iteration. Compared to the high latency of the network link, the time consumed by frequently starting and terminating the process will not become a performance bottleneck.

[0017] Furthermore, based on branch coverage and heuristic field scheduling strategies, the fields selected for this round of mutation include:

[0018] Based on the branch coverage rate, obtain the total number of new branches covered by each field, and calculate the field score for each field based on the total number of new branches covered by each field:

[0019]

[0020] Where F is a field; Score F Field score for field F; M is the proportion constant; P F For the total number of new branches covered by field F; T F The number of times field F is selected for mutation; K is the negative feedback constant.

[0021] Calculate the expected number of scheduling opportunities for each field based on the field score:

[0022]

[0023] Among them, F i For the i-th field; E(F i Score[F] is the mathematical expectation of the i-th field selected in this scheduling; i ] is F i The field scores; cnt represents the total number of fields;

[0024] Based on the mathematical expectation of each field, a preset number of variant fields are selected for this round.

[0025] The beneficial effects of the above technical solution are as follows: This solution evaluates the effectiveness of data fields by the number of new branch coverages and the number of mutations. This allows for the discovery of more branch coverage fields with less computational resources, while also reducing the interference of mutation randomness on the evaluation results. Combining mathematical expectation with the selection of mutation fields can maximize the impact of coverage feedback evaluation on scheduling, while avoiding scheduling starvation.

[0026] Furthermore, the mutation operators include flipping, arithmetic operations, special value replacement, and concatenation. The mutation operators for all fields are not completely identical. After all fields are mutated, a test case is generated.

[0027] Furthermore, methods for inserting stakes into the system under test include:

[0028] Identify all basic blocks in the program of the system under test and insert a primary key value cur_location;

[0029] The keys of the source block and the target block are XORed, and the result is mapped to a byte index in shared memory.

[0030] Each time a basic block jump is executed, the value retrieved from the shared memory based on the corresponding byte index is incremented by 1, and then the primary key value of the current block is shifted to the right to distinguish different jump directions.

[0031] The beneficial effects of the above technical solution are as follows: After instrumenting the system under test in the above manner, with the support of instrumentation, branch coverage can be obtained in the execution of each test case, so as to facilitate the dynamic updating of the field score of each field during subsequent fuzz testing.

[0032] Furthermore, methods for managing shared memory and bitmaps include:

[0033] During the initialization phase, shared memory and a bitmap are requested, and handles are obtained.

[0034] When entering fuzz testing, the shared memory handle is passed through environment variables when the system under test is started; when the system under test is tested, the hash result of the branch coverage is written to the shared memory; after the system under test is tested, the current branch coverage result is obtained from the shared memory, and then the new branch coverage is written to the global bitmap.

[0035] The beneficial effects of the above technical solution are as follows: This solution completes cross-process communication through shared memory and persistently stores the branch coverage rate found by the tested system by maintaining a global bitmap.

[0036] Furthermore, the method to determine whether a new branch is generated in this round of fuzz testing is: if the branch triggered in a single bitmap is not in the global bitmap, it indicates that a new branch has been found;

[0037] Update the global bitmap to: record the new branches discovered in this fuzz test into the global bitmap.

[0038] Secondly, a gray-box fuzz testing system for industrial control protocols is provided, comprising:

[0039] The input module is used to extract different fields and state transition relationships from industrial control protocols;

[0040] An initialization module is used to initialize shared memory and bitmaps, wherein the bitmaps include a single bitmap representing program branch information triggered by the current fuzz test and a global bitmap representing global branch information;

[0041] The mutation module is used to select the field to be mutated in this round based on the branch information in the bitmap and the heuristic field scheduling strategy, and to mutate the selected field using different mutation operators to generate test cases.

[0042] The execution module is used to start the system under test, send test samples to the instrumented system under test for testing, and shut down the system under test after the test is completed.

[0043] The monitoring module is used to monitor whether the system under test crashes during testing. If so, it records the crash information; otherwise, it obtains the branch coverage of the system under test during the test, determines whether the current fuzz test generates a new branch, updates the global bitmap and then enters the termination judgment module; otherwise, it directly enters the termination judgment module.

[0044] The termination judgment module is used to determine whether the number of iterations or the number of crashes meets the preset conditions. If so, it outputs the coverage, crash information and their corresponding test cases; otherwise, it updates the mutation count in the mutation field and updates the branch coverage when a new branch is generated, and then returns to the mutation module.

[0045] Furthermore, the monitoring module includes:

[0046] Shared memory is used to store the hash result of the branch coverage written by the system under test when the feedback information uploaded by the system under test is branch coverage.

[0047] The overlay information process is used to record newly discovered branches in the global bitmap when a branch triggered in a single bitmap is not in the global bitmap.

[0048] The crash handling module is used to receive program crash information uploaded by the system under test through the execution module, record the program crash information, and keep the test samples corresponding to the program crash information.

[0049] The beneficial effects of this invention are as follows:

[0050] (1) Improved code coverage for testing industrial control protocol programs. This invention uses compile-time instrumentation to obtain branch coverage of industrial control protocol programs, and dynamically maintains the evaluation scores of different protocol message fields through branch coverage, thereby achieving higher code coverage within the same time frame.

[0051] (2) Discover more vulnerabilities in industrial control protocol programs. Based on coverage feedback, this invention provides better evaluation results for fields that may explore more program branches, and uses heuristic algorithms to provide more selection opportunities for fields with higher scores. It can identify key fields and execute efficient scheduling strategies to discover more vulnerabilities in industrial control protocol programs. Attached Figure Description

[0052] Figure 1 This refers to the Data message of the Modbus TCP industrial control protocol.

[0053] Figure 2 This is an efficient gray-box fuzzy testing method for industrial control protocols.

[0054] Figure 3 This is a high-efficiency gray-box fuzzy testing system for industrial control protocols.

[0055] Figure 4 Explore statistics for program branches of different fields in the RTPS protocol.

[0056] Figure 5 The following are branch coverage change patterns discovered by CFSFuzz and Peach tools over 24 hours for the RTPS and IEC61850 protocols: (a) Branch coverage change patterns for the RTPS protocol; (b) Branch coverage change patterns for the IEC61850 protocol. Detailed Implementation

[0057] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0058] refer to Figure 2 , Figure 2 A gray-box fuzzy testing method for industrial control protocols is illustrated; such as Figure 1 As shown, the method includes steps S1 to S6.

[0059] In step S1, different fields and state transition relationships in the industrial control protocol are extracted. Due to the high complexity of industrial control protocols, often reaching hundreds of pages, detailed analysis of the specifications is inefficient. To construct the model file more efficiently, this solution preferably uses Wireshark to capture industrial control protocol interaction messages, thereby extracting different field information and state transition relationships.

[0060] In step S2, shared memory and bitmaps are initialized. The bitmaps include a single bitmap representing program branch information triggered by this fuzz test and a global bitmap representing global branch information.

[0061] In step S3, based on the branch information in the bitmap and the heuristic field scheduling strategy, the field to be mutated in this round is selected, and different mutation operators are used to mutate the selected field to generate test cases.

[0062] During implementation, this scheme preferably selects the following fields for this round of mutation based on branch coverage and heuristic field scheduling strategies:

[0063] Based on the branch coverage rate, obtain the total number of new branches covered by each field, and calculate the field score for each field based on the total number of new branches covered by each field:

[0064]

[0065] Where F is a field; Score F Field score for field F; M is the proportion constant; P F For the total number of new branches covered by field F; T F The number of times field F is selected for mutation; K is the negative feedback constant.

[0066] Calculate the expected number of scheduling opportunities for each field based on the field score:

[0067]

[0068] Among them, F i For the i-th field; E(F i Score[F] is the mathematical expectation of the i-th field selected in this scheduling; i ] is F i The field scores; cnt represents the total number of fields;

[0069] Based on the mathematical expectation of each field, a preset number of variant fields are selected for this round.

[0070] Specifically, this scheme preferably uses mutation operators including flipping, arithmetic operations, special value replacement, and concatenation. The mutation operators for all fields are not completely the same; after all fields are mutated, a test case is generated.

[0071] Among them, flipping refers to inverting a bit or a series of bits in the protocol field; arithmetic operation refers to performing integer addition or subtraction on a byte or a series of bytes; special value replacement refers to replacing some bits with special values, where special values ​​refer to some special numbers, such as the maximum or minimum value of the integer int; concatenation refers to randomly concatenating two protocol fields to generate new test cases.

[0072] In step S4, the system under test is started, test samples are sent to the system under test after the test is completed, and the system under test is shut down after the test is completed.

[0073] In one embodiment of the present invention, a method for inserting stakes into the system under test includes:

[0074] Identify all basic blocks in the program of the system under test and insert a primary key value cur_location;

[0075] The keys of the source block and the target block are XORed, and the result is mapped to a byte index in shared memory.

[0076] Each time a basic block jump is executed, the value retrieved from the shared memory based on the corresponding byte index is incremented by 1, and then the primary key value of the current block is shifted to the right to distinguish different jump directions.

[0077] In step S5, monitor whether the system under test crashes during the test. If so, record the crash information and proceed to step S6. Otherwise, obtain the branch coverage of the system under test during the test, determine whether the current fuzz test generates a new branch. If so, update the global bitmap and proceed to step S6. Otherwise, proceed directly to step S6.

[0078] The method to determine whether a new branch is generated in this round of fuzzing is as follows: if the branch triggered in a single bitmap is not in the global bitmap, it indicates that a new branch has been found; updating the global bitmap is to record the new branch found in this fuzzing test into the global bitmap.

[0079] The methods for managing shared memory and bitmaps include:

[0080] During the initialization phase, shared memory and a bitmap are requested, and handles are obtained.

[0081] When entering fuzz testing, the shared memory handle is passed through environment variables when the system under test is started; when the system under test is tested, the hash result of the branch coverage is written to the shared memory; after the system under test is tested, the current branch coverage result is obtained from the shared memory, and then the new branch coverage is written to the global bitmap.

[0082] Branch coverage is a key feedback information used in this gray-box fuzzing method, and its effectiveness has been proven in extensive applications. Branch coverage captures jumps in program control flow between basic blocks, and this information can be obtained using lightweight instrumentation. Since the gray-box fuzzing method runs in a different process entity than the system under test (SUT), it uses cross-process communication, i.e., shared memory, for message passing. "Noise-free" (SUT starts during testing and shuts down after testing) test execution ensures complete independence between each fuzzing loop and that the execution of each message sequence begins from the same initial server state.

[0083] In step S6, it is determined whether the number of iterations or the number of crashes meets the preset conditions. If so, the coverage, crash information and their corresponding test cases are output. Otherwise, the number of mutations in the mutation field is updated and the branch coverage is updated when a new branch is generated. Then, the process returns to step S3.

[0084] like Figure 3 As shown, this solution also provides a gray-box fuzzy testing system for industrial control protocols, which includes:

[0085] The input module is used to extract different fields and state transition relationships from industrial control protocols;

[0086] An initialization module is used to initialize shared memory and bitmaps, wherein the bitmaps include a single bitmap representing program branch information triggered by the current fuzz test and a global bitmap representing global branch information;

[0087] The mutation module is used to select the field to be mutated in this round based on the branch information in the bitmap and the heuristic field scheduling strategy, and to mutate the selected field using different mutation operators to generate test cases.

[0088] The execution module is used to start the system under test, send test samples to the instrumented system under test for testing, and shut down the system under test after the test is completed.

[0089] The monitoring module is used to monitor whether the system under test crashes during testing. If so, it records the crash information and enters the termination judgment module; otherwise, it obtains the branch coverage of the system under test during the test, determines whether the current fuzz test generates a new branch, updates the global bitmap and enters the termination judgment module; otherwise, it directly enters the termination judgment module.

[0090] The termination judgment module is used to determine whether the number of iterations or the number of crashes meets the preset conditions. If so, it outputs the coverage, crash information and their corresponding test cases; otherwise, it updates the mutation count in the mutation field and updates the branch coverage when a new branch is generated, and then returns to the mutation module.

[0091] In one embodiment of the present invention, the monitoring module includes:

[0092] Shared memory is used to store the hash result of the branch coverage written by the system under test when the feedback information uploaded by the system under test is branch coverage.

[0093] The overlay information process is used to record newly discovered branches in the global bitmap when a branch triggered in a single bitmap is not in the global bitmap.

[0094] The crash handling module is used to receive program crash information uploaded by the system under test through the execution module, record the program crash information, and keep the test samples corresponding to the program crash information.

[0095] The performance of the gray-box fuzzy testing method in this scheme is illustrated below with specific examples:

[0096] This scheme compares the growth of branch coverage by performing a certain number of random mutations on different fields of the RTPS protocol individually, and the results are as follows: Figure 4 As shown. (Through) Figure 4It can be observed that changing different fields leads to significantly different numbers of branch discoveries. Specifically, variations in the Magic Number, Flags, Length, major, and Data fields in the RTPS SPDP message resulted in an average of 606, 2048, 686, 1072, and 1726 branch coverages, respectively. The difference in branch coverage rates due to different message field selections can reach 284.8%.

[0097] Classic generative fuzzers (such as Peach) perform field-level mutations based on protocol syntax models to generate test cases. The selection of target fields plays a decisive role in whether new branches or vulnerabilities can be discovered in this round of testing.

[0098] The gray-box fuzzing method (CFSFuzz) and Peach tool in this scheme were used to explore branch coverage of the RTPS protocol and IEC61850 protocol within 24 hours. The exploration results can be used as a reference. Figure 5 (a) and (b) in the figure; compared with the Peach tool, CFSFuzz was able to improve branch coverage by an average of 7.3%-53.9% within the same 24 hours, which shows that the present invention can explore more branches of the ICS protocol procedure more quickly under the guidance of coverage feedback.

[0099] This invention designed a comparative experiment on the Fast-FTPS vulnerability discovered by Peach and CFSFuzz within 24 hours, and the analysis results are shown in Table 1.

[0100] Table 1. Statistics of Fast-FTPS vulnerabilities discovered by Peach and CFSFuzz within 24 hours.

[0101]

[0102] As shown in Table 1, within the same 24-hour period, CFSFuzz discovered four previously unknown unique vulnerabilities in the Fast-RTPS protocol, while Peach discovered only one. This indicates that CFSFuzz's use of heuristic field scheduling strategies to explore broader code coverage helps uncover more deep-seated vulnerabilities in industrial control protocol programs.

[0103] In summary, by instrumenting the industrial control protocol software under test and obtaining branch coverage information, the blindness of fuzz testing of industrial control protocols is effectively reduced. At the same time, the field scheduling algorithm implemented under the guidance of branch coverage information improves the quality of generated test cases and can efficiently discover vulnerabilities in industrial control protocols.

Claims

1. A gray-box fuzzy testing method for industrial control protocols, characterized in that, Including the following steps: S1. Extract different fields and state transition relationships from the industrial control protocol; S2. Initialize shared memory and bitmap, wherein the bitmap includes a single bitmap representing program branch information triggered by this fuzz test and a global bitmap representing global branch information; S3. Based on the branch information in the bitmap and the heuristic field scheduling strategy, select the field to be mutated in this round, use different mutation operators to mutate the selected field, and generate test cases. S4. Start the system under test, send test samples to the system under test after the instrumentation is completed, and shut down the system under test after the test is completed. S5. Monitor whether the system under test crashes during the test. If so, record the crash information and proceed to step S6. Otherwise, obtain the branch coverage of the system under test during the test and determine whether the current fuzz test generates a new branch. If so, update the global bitmap and proceed to step S6. Otherwise, proceed directly to step S6. S6. Determine whether the number of iterations or the number of crashes meets the preset conditions. If yes, output the coverage, crash information and its corresponding test cases. Otherwise, update the number of mutations in the mutation field and update the branch coverage when a new branch is generated. Then return to step S3.

2. The gray-box fuzzy testing method for industrial control protocols according to claim 1, characterized in that, Based on the branching information in the bitmap and the heuristic field scheduling strategy, the fields selected for this round of mutation include: Based on the branching information in the bitmap, obtain the total number of new branches covered by each field, and calculate the field score for each field based on the total number of new branches covered by the field: where F is a field; Score F is a field score for field F; M is a scaling constant; P F is the total number of new branches covered by field F; T F is the number of times field F was selected for mutation; K is a negative feedback constant; Calculate the expected number of scheduling opportunities for each field based on the field score: Among them, F i For the i-th field; E(F i Score[F] is the mathematical expectation of the i-th field selected in this scheduling; i ] is F i The field scores; cnt represents the total number of fields; Based on the mathematical expectation of each field, a preset number of variant fields are selected for this round.

3. The gray-box fuzzy testing method for industrial control protocols according to claim 1, characterized in that, The mutation operators include flipping, arithmetic operations, special value replacement, and concatenation. The mutation operators for all fields are not completely identical. A test case is generated after all fields are mutated.

4. The gray-box fuzzy testing method for industrial control protocols according to claim 1, characterized in that, Methods for inserting stakes into the system under test include: Identify all basic blocks in the program of the system under test and insert a primary key value cur_location; The keys of the source block and the target block are XORed, and the result is mapped to a byte index in shared memory. Each time a basic block jump is executed, the value retrieved from the shared memory based on the corresponding byte index is incremented by 1, and then the primary key value of the current block is shifted to the right to distinguish different jump directions.

5. The gray-box fuzzy testing method for industrial control protocols according to claim 1, characterized in that, Methods for managing shared memory and bitmaps include: During the initialization phase, shared memory and a bitmap are requested, and handles are obtained. When entering fuzz testing, the shared memory handle is passed through environment variables when the system under test is started; when the system under test is tested, the hash result of the branch coverage is written to the shared memory; after the system under test is tested, the current branch coverage result is obtained from the shared memory, and then the new branch coverage is written to the global bitmap.

6. The gray-box fuzzy testing method for industrial control protocols according to claim 1, characterized in that, The method to determine whether a new branch is generated in this round of fuzz testing is: if the branch triggered in a single bitmap is not in the global bitmap, it indicates that a new branch has been found; Update the global bitmap to: record the new branches discovered in this fuzz test into the global bitmap.

7. A gray-box fuzzy testing system for industrial control protocols, characterized in that, include: The input module is used to extract different fields and state transition relationships from industrial control protocols; An initialization module is used to initialize shared memory and bitmaps, wherein the bitmaps include a single bitmap representing program branch information triggered by the current fuzz test and a global bitmap representing global branch information; The mutation module is used to select the field to be mutated in this round based on the branch information in the bitmap and the heuristic field scheduling strategy, and to mutate the selected field using different mutation operators to generate test cases. The execution module is used to start the system under test, send test samples to the instrumented system under test for testing, and shut down the system under test after the test is completed. The monitoring module is used to monitor whether the system under test crashes during testing. If so, it records the crash information and proceeds to the termination judgment module. Otherwise, obtain the branch coverage of the system under test during the test, determine whether the current fuzz test has generated a new branch, if so, update the global bitmap and enter the termination judgment module; otherwise, directly enter the termination judgment module. The termination judgment module is used to determine whether the number of iterations or the number of crashes meets the preset conditions. If so, it outputs the coverage, crash information and their corresponding test cases; otherwise, it updates the mutation count in the mutation field and updates the branch coverage when a new branch is generated, and then returns to the mutation module.

8. The gray-box fuzzy testing system for industrial control protocols according to claim 7, characterized in that, The monitoring module includes: Shared memory is used to store the hash result of the branch coverage written by the system under test when the feedback information uploaded by the system under test is branch coverage. The overlay information process is used to record newly discovered branches in the global bitmap when a branch triggered in a single bitmap is not in the global bitmap. The crash handling module is used to receive program crash information uploaded by the system under test through the execution module, record the program crash information, and keep the test samples corresponding to the program crash information.

Images