Fuzz testing method, electronic device, and computer readable storage medium

Abstract

Embodiments of the present disclosure provide a fuzz testing method, an electronic device, and a computer readable storage medium. The fuzz testing method comprises: acquiring a data packet; generating a protocol rule tree model on the basis of the data packet; generating a test case on the basis of the protocol rule tree model; sending the test case to a device under test; and determining a test result on the basis of response information of the device under test.

Description

Fuzz testing method, electronic device, and computer-readable storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application CN 202311726019.X, filed on December 15, 2023, entitled “Fuzz testing method, electronic device and computer-readable storage medium,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of computer testing technology, and in particular to a fuzzy testing method, an electronic device, and a computer-readable storage medium. Background Art

[0004] Fuzz testing is a method for discovering software vulnerabilities by providing unexpected inputs to a target system and monitoring for anomalous results. Currently, test cases used in fuzz testing include basic test cases and mutation test cases. Basic test cases are generated by modeling program inputs, requiring manual learning of protocol rules and resulting in high modeling costs. Mutation test cases are generated by modifying existing test cases. While they offer high coverage and high time costs, they are unsuitable for black-box testing because they lack access to the program's internal structure and characteristics. Therefore, test cases for black-box testing are generated using a technical approach: analyzing standards, building models, and then generating test cases. However, model building requires field-by-field, byte-by-byte, or even bit-by-bit analysis of protocol fields, which is labor-intensive. Furthermore, each test case can only detect specific anomalies, resulting in low test case effectiveness.

[0005] In addition, current fuzz testing focuses on single industrial control / private protocol testing, and mainly focuses on the application layer. However, due to factors such as the complex wireless environment in the Internet of Things, the large number of related devices, and the susceptibility to interference, no reasonable, efficient, and integrated technical solutions have been proposed for the adaptation of IoT public protocols / functions (such as the fifth-generation mobile communication (5th Generation, 5G) air interface and Bluetooth, etc.) and link layers.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide a fuzz testing method, an electronic device, and a computer-readable storage medium, which can improve the scope of application of fuzz testing.

[0008] An embodiment of the present disclosure provides a fuzzy testing method, including: obtaining a data packet; generating a protocol rule tree model based on the data packet; generating a test case based on the protocol rule tree model; sending the test case to a device under test; and determining a test result based on response information of the device under test.

[0009] An embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a memory on which one or more programs are stored. When the one or more programs are executed by the one or more processors, the one or more processors implement the fuzzy testing method provided according to the embodiment of the present disclosure.

[0010] An embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon. When the program is executed by a processor, the fuzzy testing method provided according to the embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG1 is a diagram showing the composition of test hardware for implementing the fuzzy testing method provided by an embodiment of the present disclosure;

[0012] FIG2 is an architectural diagram of a fuzzy testing system according to an embodiment of the present disclosure;

[0013] FIG3 is a flow chart of a fuzzy testing method according to an embodiment of the present disclosure;

[0014] FIG4 is a flow chart of a protocol rule tree model generated by a model automatic generation algorithm according to an embodiment of the present disclosure;

[0015] FIG5 is a schematic diagram of a state update feedback loop based on a state feedback update protocol rule tree model according to an embodiment of the present application;

[0016] FIG6 is a schematic diagram of a weight update feedback loop based on a weight strategy update protocol rule tree model according to an embodiment of the present disclosure;

[0017] FIG7 is a schematic diagram of generating test cases based on a machine learning mutation strategy according to an embodiment of the present disclosure;

[0018] FIG8 is a schematic diagram of processing a test log according to an embodiment of the present disclosure;

[0019] FIG9 is a flow chart of a black-gray box hybrid fuzzy testing method according to an embodiment of the present disclosure;

[0020] FIG10 is a block diagram of an electronic device according to an embodiment of the present disclosure;

[0021] FIG. 11 is a block diagram of a computer-readable medium according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the fuzzy testing method, electronic device and computer-readable storage medium provided by the present disclosure are described in detail below with reference to the accompanying drawings.

[0023] Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of this disclosure to those skilled in the art.

[0024] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0025] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0026] The terms used herein are used only to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a," "an," and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements, and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof is not excluded.

[0027] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0028] Fuzz testing is a method for discovering software vulnerabilities by providing unexpected inputs to a target system and monitoring for unusual results. After years of development, it has become a testing tool for many technology companies.

[0029] Black box testing is a software testing method that cannot reveal the internal structure and internal characteristics of the program.

[0030] It should be noted that the embodiments of the present disclosure provide a fuzz testing method, electronic device and computer-readable storage medium that are not only suitable for black box testing, but also for gray box testing, and are applicable to multiple protocols. That is, the fuzz testing method provided by the embodiments of the present disclosure is a black-gray box hybrid fuzz test that can be applied to multiple protocols.

[0031] FIG1 is a composition diagram of the test hardware for implementing the fuzz testing method provided by an embodiment of the present disclosure.

[0032] As shown in Figure 1, the test hardware for implementing the fuzz testing method includes a test device 10, a device under test 20 and test hardware 30. The test device 10 can be a computer (PC), and the software deployed in the test device 10 includes but is not limited to the interactive user interface (User Interface, referred to as UI) of the test tool, a test environment plug-in and software code, etc. The test device 10 can be a Linux kernel device, or it can be deployed and built in the form of a virtual machine / container (Docker). The device under test 20 is a device that needs to test a protocol / function, which can be a mobile device, such as a mobile phone, pad or computer. The test protocol includes but is not limited to protocols such as those applicable to base stations, the Internet of Things and smart phones.

[0033] The test device 10 and the device under test 20 need to be connected through the test hardware 30, and the test hardware 30 needs to be set according to the test protocol. For example, if the wireless local area network universal standard (IEEE 802.11) protocol is tested, the test hardware 30 is a wireless network card that supports monitoring mode; if the 5G air interface protocol is tested, the test hardware 30 is a deeply customized 5G fuzz terminal; if the Bluetooth protocol is tested, the test hardware 30 is a Bluetooth adapter / Bluetooth Low Energy (BLE) adapter; if the Near Field Communication (NFC) protocol is tested, the test hardware 30 is an NFC adapter, connecting the mobile phone to the PC to obtain log information to assist in monitoring; if the robustness of the instruction set (Attention, referred to as AT) command of the device under test 20 is tested, it is only necessary to directly connect the Universal Serial Bus (USB) to the serial port of the device under test 20. The test protocol may also be AT protocol, Secure Shell Protocol (SSH), Hypertext Transfer Protocol (HTTP), etc., which are not listed here one by one.

[0034] The test device 10 and the test hardware 30 can be connected via a USB or an Ethernet port.

[0035] The test equipment 10 includes but is not limited to most devices that support 5G air interface, 802.11, Bluetooth, BLE, NFC, AT, SSH, HTTP and other protocols, including but not limited to 5G base stations, smart phones, laptops, Bluetooth headsets / speakers, Bluetooth mice / keyboards, routers, wireless network cards, NFC smart devices, IoT smart ecological home products and WEB websites, etc.

[0036] FIG2 is an architectural diagram of a fuzzy testing system according to an embodiment of the present disclosure.

[0037] As shown in FIG2 , the fuzzy testing system includes a testing device 10 and a device under test 20 , and adopts a browser / server (B / S) architecture.

[0038] The test device 10 and the device under test 20 are connected via a wireless network card, USB or serial port. The test device 10 includes a front-end UI 11 and a back-end server 12, and the front-end UI 11 and the back-end server 12 are connected via middleware signals. The front-end UI 11 is used for display and assists users in implementing fuzz testing, such as message uploading, configuration management, log reporting and status display. The back-end server 12 interacts with the device under test 20 to implement fuzz testing. The back-end server 12 mainly includes a model generation module, a model import module, a configuration module, a monitoring module, a log module and a report module, which are used to implement functions such as data mutation, test case generation, test status monitoring and model updating, and test weight optimization. The device under test 20 can be used to test 802.11 protocols, Bluetooth protocols, NFC protocols, AT protocols and abnormality / status monitoring. The middleware can be a serial port.

[0039] In the embodiment of the present disclosure, the front-end UI 11 can be written using the Django framework and the Vue language, and the back-end server 12 can be written using the Python language.

[0040] Based on the above-mentioned testing hardware and fuzz testing system, the fuzz testing method described below is implemented. This fuzz testing method can be used to discover security vulnerabilities in the device under test, thereby improving the security of the device under test.

[0041] An embodiment of the present disclosure provides a fuzzy testing method.

[0042] FIG3 is a flowchart of a fuzzy testing method according to an embodiment of the present disclosure.

[0043] As shown in FIG3 , the fuzzy testing method provided by the embodiment of the present disclosure includes the following steps S301 to S305 .

[0044] In step S301, a data message is obtained.

[0045] Data packets can be captured using software such as Wireshark or tcpdump. Data packets can also be captured using network packet capture libraries such as libpcap or WinPcap. By analyzing data packets, information such as protocol functionality, encryption protocols, and fields can be determined.

[0046] In some embodiments, the data message comes from the device under test, or may be generated in advance by other means. The embodiments of the present disclosure do not limit the format of the data message, for example, the format of the data message may be a pcap format.

[0047] In step S302, a protocol rule tree model is generated based on the data message.

[0048] The protocol rule tree model is a tree model built based on the information in the data message.

[0049] In some embodiments, generating a protocol rule tree model based on the data message (ie, step S302 ) includes: extracting features from the data message to obtain protocol features; and generating a protocol rule tree model based on the protocol features.

[0050] The protocol rule tree model in this embodiment is established based on protocol features. Compared with the protocol rule tree model established based on feedback information from the device under test, the generated test cases are applicable not only to application layer protocols but also to underlying protocols, thus expanding the test scope.

[0051] To obtain more accurate protocol features, we can refer to the protocol used by the datagram when extracting protocol features. Specifically, we can refer to the protocol configuration parameters for feature extraction. Protocol configuration parameters are common to the protocol. For example, for the Wi-Fi protocol, these parameters include, but are not limited to, protocol field length and protocol field type.

[0052] In some embodiments, the step of extracting features from data packets to obtain protocol features includes: obtaining protocol configuration parameters corresponding to the data packets; and extracting features from the data packets based on the protocol configuration parameters to obtain protocol features.

[0053] After the test device is connected to the device under test, the protocol configuration parameters are obtained by enumerating (traversing) the functions supported by the device under test. The protocol features in the data message are then extracted using a feature extraction algorithm, and the protocol configuration parameters are referenced when extracting the protocol features.

[0054] The disclosed embodiment extracts features from data packets through a Longest Common Subsequence (LCS) algorithm to obtain protocol features.

[0055] The LCS algorithm can extract the largest common substring of two strings and calculate the similarity of two strings. The LCS algorithm can find the largest common substring of string A and string B by exhaustively comparing all subsequences. n-1 , or string A n-1 The maximum common substring with string B is found recursively. If C[i,j] is the length of a common substring of sequence Xi and Yi, then the optimal common substring of LCS algorithm is obtained.

[0056] The specific steps of the LCS algorithm are as follows:

[0057] Construct a table and input A{1,2,…,I} and B{1,2,…,J} into the table, where I and J are integers greater than 1.

[0058] When filling out the table, proceed from left to right and top to bottom. The information is divided into two parts: the first is the value of C[i,j]—that is, the length of the largest common substring of cells so far—and the second is the cell from which this C[i,j] value originates. Then, backtracking from the bottom right of the table, we find the largest common substring of the LCS substrings in reverse order.

[0059] After calculating the length of the largest common substring, the similarity between the strings is calculated. The similarity between the strings is the length of the largest common substring of the two strings divided by the average length of the two strings. After simplification, we get formula (1):

[0060] In formula (1), w1 represents the first string, w2 represents the second string, len() is a function for calculating the length of a string, and simw(w1, w2) represents the similarity function between the first string and the second string.

[0061] In the embodiment of the present disclosure, each data packet can be regarded as a character string, and the similarity and common character string between the data packet and the protocol configuration parameter can be regarded as a protocol feature.

[0062] The LCS algorithm is used to obtain the maximum common substring in the protocol configuration parameters and data packets, thereby obtaining the protocol characteristics.

[0063] In some embodiments, generating a protocol rule tree model based on the data message (ie, step S302 ) includes: parsing the data message to obtain an analysis result; and obtaining a protocol rule tree model based on the analysis result.

[0064] In the embodiment of the present disclosure, data packets can be analyzed by tools such as Wireshark to obtain analysis results.

[0065] In some embodiments, the analysis result includes at least one of the hierarchical structure of the data message, the position of the fields, the association relationship between the fields, the field type and the field length.

[0066] The analysis results are used to build a model, and then a protocol rule tree model is obtained through a model automatic generation algorithm. The embodiment of the present disclosure does not limit the model automatic generation algorithm. For example, the model automatic generation algorithm can adopt an algorithm in the field of artificial intelligence.

[0067] FIG4 is a flow chart of generating a protocol rule tree model using a model automatic generation algorithm according to an embodiment of the present disclosure.

[0068] As shown in Figure 4, the data message is analyzed by the Wireshark tool to obtain features such as the message layer structure, field position, field type, field length, and the relationship between fields. Then, these features are used to generate a protocol rule tree model through the model automatic generation algorithm, and the protocol rule tree model is stored in the protocol rule tree model pool.

[0069] In step S303, a test case is generated based on the protocol rule tree model.

[0070] Because the protocol rule tree model is generated based on datagram characteristics, it better meets protocol requirements, and the test cases generated from it help improve test accuracy. Furthermore, the protocol rule tree model, and thus the test cases, are automatically generated based on the datagrams, eliminating the need for manual intervention and achieving a 99.99% efficiency in test case generation.

[0071] In order to improve the effectiveness of test cases, the embodiment of the present disclosure utilizes a protocol rule tree model combined with a mutation strategy to generate test cases.

[0072] In some embodiments, generating test cases based on the protocol rule tree model (ie, step S303 ) includes: generating test cases based on the protocol rule tree model and a mutation strategy.

[0073] Mutation strategies are used to generate and adjust test cases. The disclosed embodiments can mutate protocol field features, mutate based on the target response state of the device under test, or utilize machine learning algorithms. Test cases are generated based on the protocol rule tree model and mutation strategies. The mutation strategies significantly increase the number of abnormal responses in test cases, thereby improving the effectiveness of test cases and, in turn, the efficiency of fuzz testing.

[0074] In some embodiments, the mutation strategy includes a field mutation strategy, wherein the field mutation strategy refers to a strategy for performing field mutation processing on protocol field features in a protocol rule tree model.

[0075] Test cases can be obtained by combing through the fields in the network file that may have problems, querying the standard protocol and correcting the problematic fields.

[0076] In some embodiments, the field mutation strategy includes at least one of a random (s_random) mutation strategy and a bit flip (s_bytes) mutation strategy, wherein the random mutation strategy is a strategy for randomly mutating the protocol field characteristics in the protocol rule tree model, such as changing one or several fields, and the bit flip mutation strategy is a strategy for flipping the protocol field characteristics in the protocol rule tree model, such as flipping the order of the fields.

[0077] The disclosed embodiments can also design a variation strategy based on the target response state of the device under test. Variation strategies include state feedback strategies, which are generated based on the target response state of the device under test. The device under test can provide feedback on the target response state via a state monitor.

[0078] In some embodiments, the step of generating the state feedback strategy includes: acquiring new state data of the device under test; and generating the state feedback strategy based on the new state data.

[0079] When a new test state appears on the device under test, a state feedback strategy is generated based on the new test state, and the protocol rule tree model in the model pool is expanded, forming a state update feedback loop. During the fuzz testing process, the model pool is continuously updated and expanded to improve the effectiveness of fuzz testing.

[0080] FIG5 is a schematic diagram of a state update feedback loop based on a state feedback update protocol rule tree model according to an embodiment of the present application.

[0081] As shown in Figure 5, the fuzz tester extracts the protocol rule tree model from the model pool and generates a mutation message (test case) based on the protocol rule tree model, and then sends the mutation message to the device under test. The state monitor monitors the state of the device under test and returns the monitoring message to the fuzz tester. When the state monitor finds that the test case leads to a new state, it sends the new state to the state machine. At the same time, it analyzes the mutation field of the test case, generates a new protocol rule tree model, and stores the new protocol rule tree model in the model pool, expanding the number of protocol rule tree models in the model pool.

[0082] In practical applications, according to the target response state in the first round of testing, the new state of the device under test is extracted, and a new mutation strategy is generated based on the new state, thereby generating new test cases.

[0083] In some embodiments, the mutation strategy includes a weight feedback strategy, which uses a weight feedback algorithm to generate a protocol rule tree model based on the test case response time and the probe response time. The weight feedback algorithm mutates the fields with heavier weights in the protocol rule tree model to obtain test cases, thereby improving the directionality of the test cases, thereby reducing the number of test cases and improving the effectiveness of the test cases.

[0084] The steps for generating a weighted feedback strategy include: determining a target abnormal state from a target response state; extracting the test case response time of the target abnormal state; and using the test case response time and the probe response time as weights to generate a weighted feedback strategy, wherein the probe response time is the response time of the device under test to a non-mutation test case.

[0085] After the test case is sent to the device under test, the response time of the device under test for the test case is recorded, that is, the test case response time is obtained. A non-mutation test case is sent to the device under test, and the response time of the device under test for the non-mutation test case is recorded, that is, the response time of the non-mutation test case is obtained. The weighted feedback algorithm uses the probe response time and the test case response time as weights to establish a test case weight configuration tree model. After multiple rounds of testing, the weights of the test case weight configuration tree model are dynamically adjusted to update the proportion of the protocol rule tree model corresponding to a specific test case in the model pool, thereby reducing the number of test cases and improving the effectiveness of the test cases.

[0086] FIG6 is a schematic diagram of a weight update feedback loop based on a weight strategy update protocol rule tree model according to an embodiment of the present disclosure.

[0087] As shown in Figure 6, during the test process, the status monitor monitors the status of the device under test. When the status monitor finds that the response time is abnormal, it uses the test case response time and the probe response time to adjust the weight of the protocol rule tree model corresponding to the test case, and updates the proportion of the protocol rule tree model corresponding to the test case in the model pool.

[0088] It should be noted that the steps in the state update feedback loop in FIG6 are the same as those in FIG5 and will not be described in detail here.

[0089] In practical applications, according to the target abnormal state in the first round of testing, the test case response time and the probe response time are used as weights, and a weighted feedback algorithm is used to generate a protocol rule tree model based on the test case response time and the probe response time, thereby generating new test cases.

[0090] In some embodiments, the mutation strategy includes a machine learning mutation strategy, which is a mutation strategy obtained through machine learning based on valid test cases and data packets, wherein the valid test case refers to a test case that can be fed back by the device under test.

[0091] The valid test cases (for example, 1,000 valid test cases) and network packets from the first round of testing are used as training data for machine learning. A mutation strategy is obtained through machine learning. The machine learning model used can be a recurrent neural network (RNN) encoder-decoder (Seq2Seq) model. The trained machine learning model can directly generate new test cases.

[0092] The Seq2Seq model, using an RNN, performs sequential predictions, where one input sequence corresponds to one output sequence. The valid test cases from the first round of user testing are arranged in chronological order to form the input sequence, and the test cases after the user's input node are arranged in chronological order to form the output sequence. The Seq2Seq model is then used for predictions. A dataset is considered satisfactory when the training samples meet the following conditions. The Seq2Seq model is then trained using this dataset using unsupervised training, with network packets input to generate test cases. Upon completion of training, a machine learning model is obtained.

[0093] The conditions met by the training samples in the dataset include but are not limited to: the number of samples in the dataset reaches a preset training threshold; the training samples in the dataset cover more edges (the required range); the training samples in the dataset contain all changes that meet the required state; and the training samples in the dataset meet the original fuzzy tool.

[0094] FIG7 is a schematic diagram of generating test cases based on a machine learning mutation strategy according to an embodiment of the present disclosure.

[0095] As shown in Figure 7, the machine learning model obtains training samples from the message request and response pool to train the machine learning model. Once the machine learning model training is complete, it is used to generate mutated messages. The fuzz tester sends the mutated messages to the device under test for fuzz testing, generating valid request and response messages. These fuzzed test cases (data packets) are injected into the network and the device under test's responses are recorded to assess its robustness to malformed inputs. At the same time, non-mutated original network messages can be sent to monitor the response messages for supplementary monitoring.

[0096] The disclosed embodiment combines multiple mutation strategies such as state feedback, weight feedback, and machine learning. During the test process, the mutation strategy is dynamically adjusted according to the state feedback from the state monitor, thereby achieving dynamic generation of test cases and improving the effectiveness of test cases.

[0097] In step S304, the test case is sent to the device under test.

[0098] After the test cases are generated by the fuzz tester, they are sent to the device under test.

[0099] In some embodiments, the protocol rule tree model includes multiple nodes, and the test case is executed sequentially on the multiple nodes. The fuzz testing method further includes: monitoring the execution process of the test case to obtain input data; determining the current node of the protocol rule tree model based on the input data; and determining the next node and corresponding data message based on the current node until an abnormality occurs in the target response state or the protocol rule tree model traversal is completed.

[0100] Input data refers to the data of the input node. The input data is input into the current node. The current node processes and outputs a data message. The data message is input into the next node and executed sequentially until an exception occurs in the target response state or the protocol rule tree model traversal is completed.

[0101] Specifically, the device under test invokes a status monitor to monitor the device's response status. Based on this response status, the device monitors the process and results of executing the test case. For example, techniques such as taint analysis and execution flow tracing are used to monitor the execution process and results. This information is fed back to the traffic identification module to determine which node in the protocol rule tree model the device under test has reached, thereby determining the next node's message generation. This process continues until a target response anomaly occurs or the protocol rule tree model is traversed completely.

[0102] Taint analysis is a technique that tracks and analyzes the flow of tainted information within a program. It uses taint analysis to mark abnormal input data as tainted data. Then, by tracking the flow of information related to the tainted data, it determines which node in the protocol rule tree the data enters, and thus determines the next node to generate the message, until the target response anomaly occurs.

[0103] Flow tracing technology can track modules, threads, functions, and thread types. It can record the function call process, the time point when the function is called, and the line number of the source file where it is located. It can also obtain the parameter values ​​passed in when calling the function.

[0104] In some embodiments, due to the high noise levels of test messages in the environment, when the test equipment calls the status monitor to monitor the response status of the device under test in real time, it is necessary to remove noise from the target response status, collect information such as the test case's response time, response status, response message length, and response anomalies, and record them in a log. Non-mutated probe messages can also be sent to monitor the response messages as a supplementary monitoring measure.

[0105] In step S305 , the test result is determined based on the response information of the device under test.

[0106] If the response information of the device under test is normal, the test result is passed. If the response information of the device under test is abnormal, the test result is a vulnerability.

[0107] In some embodiments, the fuzz testing method further includes: recording the data message that caused the fault and the test log when the test is completed or a vulnerability is discovered; and generating a visual test report based on the data message that caused the fault and the test log.

[0108] In some embodiments, after generating a visual test report based on the data message generating the fault and the test log, the method further includes: reproducing and locating the vulnerability based on the visual test report.

[0109] Recording the data packets and test logs that cause failures helps reproduce detected security vulnerabilities, facilitating developer location and remediation. Data packets and test logs can be stored in the task management module, where logs and reports provide access to complete test cases and test information. The test case reproduction module can reproduce vulnerabilities, allowing developers to locate them simply by reproducing the test case with the anomaly.

[0110] The test log contains exception logs. By analyzing the exception logs, the visualization results can be used for intuitive analysis, such as the change of test response time with test time, the correlation between test requests and test responses, and the comparison of test model effects. The visualization conclusions can be used to assist in problem analysis and location.

[0111] FIG8 is a schematic diagram of processing a test log according to an embodiment of the present disclosure.

[0112] As shown in Figure 8, based on the test log, network messages can be extracted, vulnerabilities can be reproduced, test reports can be exported, and problems can be located. Problems can be identified by analyzing the test report.

[0113] The fuzz testing method provided by the disclosed embodiments generates fuzzed test cases based on a protocol rule tree model. Because the protocol rule tree model is generated based on data packets, the test cases can capture protocol characteristics. Therefore, the test cases are applicable not only to application-layer protocols but also to underlying protocols, expanding the scope of fuzz testing. Furthermore, because both the protocol rule tree model and the test cases are automatically generated, significant labor costs can be saved.

[0114] In order to more clearly understand this embodiment, the black-gray box hybrid fuzz testing method is taken as an example for description below.

[0115] FIG9 is a flowchart of a black-gray box hybrid fuzzy testing method according to an embodiment of the present disclosure.

[0116] As shown in FIG9 , the black-gray box hybrid fuzz testing method includes steps S901 to S919 .

[0117] In step S901, the device under test and the protocol are determined.

[0118] In step S902, a new test task is created and parameters of the test task are configured.

[0119] In step S903, it is determined whether a protocol rule tree model exists. If not, step S904 is executed; if so, step S906 is executed.

[0120] In step S904, the data message is captured and the protocol features are extracted.

[0121] Use the Wireshark tool to capture data packets and extract features from the data packets to obtain protocol features.

[0122] In step S905, a protocol rule tree model is established based on the protocol characteristics.

[0123] A protocol rule tree model is established based on protocol features.

[0124] In step S906, the captured data message is uploaded.

[0125] In step S907, the model automatic generation algorithm establishes a protocol rule tree model.

[0126] The data message is used to determine the model automatic generation algorithm, and a protocol rule tree model is established based on the protocol characteristics and the model automatic generation algorithm.

[0127] In step S908, a test case is generated.

[0128] Generate test cases using the protocol rule tree model. When the state feedback strategy, weight feedback strategy, and machine learning mutation strategy are obtained, generate test cases based on the protocol rule tree model and mutation strategy.

[0129] In step S909, the test case is sent to the device under test.

[0130] Use the underlying packet sending module to send test cases to the device under test.

[0131] In step S910 , the state monitor monitors the target response state of the device under test.

[0132] In step S911, determine whether a new state has appeared. If not, execute step S912; if so, execute steps S916 and S919.

[0133] In step S912, determine whether a target abnormal state occurs; if not, execute step S913; if so, execute steps S917 and S919.

[0134] In step S913, determine whether it has machine learning function, if not, execute step S914; if so, execute step S918.

[0135] In step S914, it is determined whether the test task is completed. If so, step S915 is executed; if not, the process returns to step S905 to continue executing the test task.

[0136] In step S915 , a test report is generated and exported.

[0137] In step S916, the new state is fed back to the test case generation module.

[0138] Generate new test cases based on the state feedback strategy. The specific method of generating new test cases based on the state feedback strategy can be found in step S303, which will not be described in detail here.

[0139] In step S917, the weights are fed back to the test case generation module.

[0140] Generate new test cases based on the weighted feedback strategy. The specific method of generating new test cases based on the weighted feedback strategy can be found in step S303 and will not be described in detail here.

[0141] In step S918, the machine learning model is trained.

[0142] Use test messages in the target abnormal state to train the machine learning model, and use the machine learning model to obtain new test cases.

[0143] In step S919, the data message and the test log in which the abnormal state occurs are recorded.

[0144] Record abnormal data packets and test logs to facilitate the generation of visual test reports, as well as subsequent vulnerability reproduction and location.

[0145] 10 , an embodiment of the present disclosure provides an electronic device, comprising: one or more processors 1001; a memory 1002 on which one or more programs are stored, and when the one or more programs are executed by the one or more processors 1001, the one or more processors 1001 implement any one of the above-mentioned fuzzy testing methods.

[0146] The electronic device according to the embodiment of the present disclosure may further include one or more I / O interfaces 1003 connected between the processor 1001 and the memory 1002 and configured to implement information interaction between the processor 1001 and the memory 1002 .

[0147] The processor 1001 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 1002 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) 1003 is connected between the processor 1001 and the memory 1002, and can realize information interaction between the processor 1001 and the memory 1002, including but not limited to a data bus (Bus), etc.

[0148] In some embodiments, the processor 1001 , the memory 1002 , and the I / O interface 1003 are connected to each other via a bus 1004 , and further connected to other components of the computing device.

[0149] 11 , an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements any of the above-mentioned fuzzy testing methods when the program is executed by a processor.

[0150] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0151] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A fuzz testing method, comprising: Get data packets; Generate a protocol rule tree model based on the data message; Generate test cases based on the protocol rule tree model; Sending the test case to the device under test; A test result is determined based on the response information of the device under test.

2. The method according to claim 1, wherein: Generating a protocol rule tree model based on the data message includes: Extracting features from the data message to obtain protocol features; The protocol rule tree model is generated based on the protocol features.

3. The method according to claim 2, wherein: Extracting features from the data message to obtain protocol features includes: Obtaining protocol configuration parameters corresponding to the data message; Feature extraction is performed on the data message based on the protocol configuration parameters to obtain protocol features.

4. The method according to claim 1, wherein: Generating a protocol rule tree model based on the data message includes: Parsing the data message to obtain an analysis result; The protocol rule tree model is obtained based on the analysis result.

5. The method according to claim 4, wherein: The analysis result includes at least one of the hierarchical structure, field position, association relationship between fields, field type and field length of the data message.

6. The method according to claim 1, wherein: Generating test cases based on the protocol rule tree model includes: The test case is generated based on the protocol rule tree model and mutation strategy.

7. The method according to claim 6, wherein: The mutation strategy includes a field mutation strategy, and the field mutation strategy refers to a strategy for performing field mutation processing on the protocol field features in the protocol rule tree model.

8. The method according to claim 7, wherein: The field mutation strategy includes: at least one of a random mutation strategy and a bit flip mutation strategy; Wherein, the random mutation strategy is a strategy for performing random mutation processing on the protocol field features in the protocol rule tree model; The bit flip mutation strategy is a strategy for flipping the protocol field features in the protocol rule tree model.

9. The method according to claim 6, wherein: The mutation strategy includes a state feedback strategy, which is a strategy generated based on the target response state of the device under test.

10. The method according to claim 9, wherein: The step of generating the state feedback strategy comprises: Obtaining new status data of the operation of the device under test; The state feedback strategy is generated based on the new state data.

11. The method according to claim 6, wherein: The mutation strategy includes a weight feedback strategy, and the steps of generating the weight feedback strategy include: determining a target abnormal state from a target response state of the device under test; Extracting the test case response time of the target abnormal state; Taking the test case response time and the probe response time as weights, generating the weight feedback strategy, The probe response time is the response time of the device under test to the non-mutation test case. Response time.

12. The method according to claim 6, wherein: The mutation strategy includes a machine learning mutation strategy, which is a mutation strategy obtained through machine learning based on valid test cases and the data packets, wherein the valid test cases refer to test cases that can be fed back by the device under test.

13. The method according to claim 1, wherein: The protocol rule tree model includes a plurality of nodes, the test case is executed sequentially on the plurality of nodes, and the method further includes: Monitor the execution process of the test case and obtain input data; Determine a current node of the protocol rule tree model based on the input data; The next node and the corresponding data message are determined based on the current node until an abnormality occurs in the target response state of the device under test or the traversal of the protocol rule tree model is completed.

14. The method according to claim 1, further comprising: When the test is completed or a vulnerability is found, the data message that caused the fault and the test log are recorded; A visual test report is generated based on the data message generating the fault and the test log.

15. The method according to claim 14, wherein: After generating a visual test report based on the data message generating the fault and the test log, the method includes: Vulnerability reproduction and vulnerability location are performed based on the visual test report.

16. An electronic device, comprising: one or more processors; A memory having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the A fuzz testing method as claimed in any one of claims 1 to 15.

17. A computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the fuzz testing method according to any one of claims 1 to 15.

Images