A neural network gradient information-oriented fuzzing method and device

Abstract

The embodiment of the specification discloses a kind of fuzzy testing methods based on neural network gradient information orientation, the method comprises: monitoring the speed of fuzzy testing to the program to be tested, whether it needs to update test case by training neural network model according to fuzzy testing speed determination;According to the fuzzy testing operation that has been carried out, the training data set used for training neural network model is determined, neural network model training is carried out using the training data set, and the target model after training is obtained;Using the target model, the gradient information is generated by predicting and processing test case file to be processed;For any type of test case file to be processed, determine the processing strategy corresponding to the type of test case file to be processed, according to the processing strategy, the gradient information is used to process the type of test case file to be processed, and the target test case file is obtained;Using the target test case file, the program to be tested is carried out fuzzy testing.

Description

Technical Field

[0001] This application relates to the field of fuzz testing technology, and in particular to a fuzz testing method and apparatus based on neural network gradient information guidance. Background Technology

[0002] Fuzzing is a method for discovering software vulnerabilities by providing unexpected input to a target system and monitoring the abnormal results. Fuzzing is a simple technique that can uncover many high-risk vulnerabilities in a program. It can not only locate these vulnerabilities before the software goes live, but also alert users to potential vulnerability attacks. Fuzzing uses a large amount of semi-valid data to identify security vulnerabilities in an application through program anomalies. "Semi-valid" data refers to data that an application must identify where some and most of the data is correct so that the tested program will treat it as valid data. In this process, other parts are invalid, leading to application errors, crashes, or security issues.

[0003] The idea behind fuzzing is to trigger unexpected vulnerabilities in a program through illogical and disordered input. Adding strategies or methods to these disordered inputs can guide the fuzzing input mutations in a desired direction, thereby improving fuzzing efficiency. Deep learning is primarily used in the seed mutation stage of fuzzing. The quality of the input seed in fuzzing greatly affects testing efficiency. Deep learning can identify common characteristics of input files that lead to higher coverage, more crashes, or more specific execution paths. Based on this feature information obtained from neural networks, the likelihood of triggering corresponding seed generation and seed mutation will increase.

[0004] One strategy for optimizing numerous fuzzing techniques is to guide seed mutation through feedback information during program execution. For example, the American Fuzzing Loop (AFL) maintains an input queue for branch coverage information and optimizes fuzzing execution by evaluating the coverage of new branches and providing feedback on execution efficiency. However, this heuristic feedback mechanism sometimes struggles to cover certain less prominent but crucial branches, leading to increasing research proposing improvements to this dynamic feedback mechanism for fuzzing.

[0005] Mutation-based fuzzing is the simplest and most direct technique from a data generation perspective. Because it doesn't need to consider any data format limitations, it generates data very quickly. However, this method also leads to problems such as significant data invalidation and duplication. Many existing techniques aim to improve the efficiency of mutation-based fuzzing by reducing invalid data, and branch-information guided fuzzing is one such solution. Branch-information guided fuzzing (also known as coverage-information guided fuzzing) combines random data input with internal program execution path information and coverage information, optimizing data mutation through program branch information feedback. The principle of branch-information guided fuzzing tools is as follows: Figure 1 As shown. Summary of the Invention

[0006] This specification provides a fuzz testing method and apparatus based on neural network gradient information guidance to solve the technical problem of how to improve the effect and efficiency of fuzz testing.

[0007] To address the aforementioned technical problems, the embodiments in this specification provide the following technical solutions: This specification provides an embodiment of a fuzz testing method based on neural network gradient information guidance, the method comprising: Monitor the fuzzing speed of the program under test, and determine whether it is necessary to update the test cases by training a neural network model based on the fuzzing speed; The training dataset used to train the neural network model is determined based on the fuzzy testing operations already performed. The neural network model is then trained using the training dataset to obtain the trained target model. The target model is used to predict and generate gradient information for the test case files to be processed. For any type of test case file to be processed, determine the processing strategy corresponding to that type of test case file, and process the test case file of that type using the gradient information according to the processing strategy to obtain the target test case file; The program under test is fuzz-tested using the target test case file. The training dataset used to train the neural network model, determined based on the fuzzing operations already performed, includes: The relationships between different branches of the tested program are determined based on the fuzzing operations already performed. Identify candidate branches that meet the rarity condition, and select branches of interest that are independent of each other from the candidate branches based on the relationship between the different branches. Test cases that can cover the branches of interest will be used as training data in the training dataset used to train the neural network model.

[0008] Preferably, the test case files to be processed include test case files that did not participate in neural network training, and the processing strategy for determining the test case files that did not participate in neural network training includes: In test case files that did not participate in neural network training, the bytes with predicted labels of specific values ​​are mutated.

[0009] Preferably, the feature value is 0.

[0010] Preferably, the test case files to be processed include test case files that hit the branch of interest, and the processing strategy for determining the test case files that did not participate in neural network training includes: The gradient information is used to protect the bytes in the test case files that did not participate in the neural network training, causing the branch of interest to be hit, and mutating other bytes.

[0011] Preferably, byte mutation includes: Perform integer addition, subtraction, bitwise flipping, and / or random mutation on bytes.

[0012] Preferably, before training the neural network model using the training dataset, the method further includes: The number of positive and negative training data in the training dataset is balanced.

[0013] Preferably, balancing the number of positive and negative training data in the training dataset includes: If a test case file hits a branch of interest in the program under test, then the bytes at each position in the test case file are mutated respectively; After each mutation, it is determined whether the mutated test case file can hit the branch of interest in order to determine the key position; wherein, the key position is the position associated with the branch of interest. The bytes in the non-critical positions of the test case file are mutated, and the resulting new test case file is used as training data in the training dataset.

[0014] Preferably, the method further includes: Use MFB as the marker for the stake.

[0015] Preferably, the method further includes: Instrumentation is performed using LLVM's Pass compiler structuring techniques.

[0016] This specification provides an embodiment of a fuzz testing device based on neural network gradient information guidance, the device comprising: The runtime status monitoring module is used to monitor the fuzzing speed of the program under test and determine whether it is necessary to update the test cases by training a neural network model based on the fuzzing speed. The test update module is used to determine the training dataset used to train the neural network model based on the fuzz test operations that have been performed, and to train the neural network model using the training dataset to obtain the trained target model. Furthermore, the target model is used to predict and generate gradient information for the test case files to be processed; Furthermore, for any type of test case file to be processed, determine the processing strategy corresponding to that type of test case file, and process the test case file to be processed using the gradient information according to the processing strategy to obtain the target test case file; And, fuzz testing is performed on the program under test using the target test case file; The training dataset used to train the neural network model, determined based on the fuzzing operations already performed, includes: The relationships between different branches of the tested program are determined based on the fuzzing operations already performed. Identify candidate branches that meet the rarity condition, and select branches of interest that are independent of each other from the candidate branches based on the relationship between the different branches. Test cases that can cover the branches of interest will be used as training data in the training dataset used to train the neural network model.

[0017] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: The aforementioned technical solution records, checks the reliability of, and filters test cases for rare branches and their corresponding coverage branches during the execution of test cases in the program under test, and continuously updates the relevant information of the rarest and independent branches. When monitoring finds it difficult to find a new program execution path, it runs neural network training, generates new test cases through gradient information mutation of the neural network, and then uses the new test cases as initial seeds for fuzz testing, thereby effectively improving the effect and efficiency of fuzz testing. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments of this specification or the prior art will be briefly described below. Obviously, the drawings used in some embodiments of this application are only described below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is a schematic diagram illustrating the principle of a branch-information-driven fuzz testing tool.

[0020] Figure 2 This is a schematic diagram of the research process for deep learning models.

[0021] Figure 3 This is a schematic diagram of the experimental results of an information-oriented fuzz testing tool based on deep learning.

[0022] Figure 4 This is a schematic diagram of the workflow of the fuzz testing tool provided in the first embodiment of this specification.

[0023] Figure 5 This is a schematic diagram of test case updates in the first embodiment of this specification. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments involved in the specific implementation are only a part of the embodiments of this application, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments in the specific implementation without creative effort should fall within the protection scope of this application.

[0025] To better demonstrate the impact of deep learning models on coverage-oriented fuzzy testing, Example 1 presents a systematic and extensive empirical study of four deep learning models across six relevant projects. The research process is as follows: Figure 2 As shown. The specific research content is explained below: 1. Define your research motivation In recent years, a significant amount of work in the field of fuzzing has been related to deep learning, much of which involves using neural networks to learn the relationship between input seed test cases and the internal branch coverage of a program. A novel aspect of these approaches is their use of existing models from computer vision or natural language processing (e.g., fully connected networks, LSTM, and GANs) to predict the path behavior during program execution. While deep learning-based fuzzing has achieved success, it remains unclear how well these models perform in fitting the behavior of discontinuous program operations (e.g., modulo operations, function checks, string inclusion checks, etc.).

[0026] Moreover, compared to the images and sentences in its original dataset, the software has a more diverse range of program branches.

[0027] For example, executing one branch might only require satisfying one constraint (shallow branch), while another branch might require satisfying more than 10 constraints (deep branch). Therefore, it's unclear whether a unified deep learning model can fully express all branch behaviors. Many inherent problems of deep learning models (e.g., overfitting, training data quality) have not been widely studied. For fuzzing tools employing deep learning, in addition to horizontal performance comparisons with each other, vertical performance comparisons with traditional fuzzing tools in software engineering are also necessary. Furthermore, a common set of code coverage test instrumentation statistics and coverage comparison standards needs to be established for performance testing of different tools to eliminate performance comparison errors caused by different test statistics methods, enabling a truly meaningful comparison of the performance of various fuzzing tools. Because fuzzing test cases are characterized by large-scale generation and rapid testing, efficiency becomes the most important metric for fuzzing tools. Typically, fuzzing tools can be divided into two parts based on their internal principles: generating test cases and actual testing. However, fuzzing tools combined with deep learning also need to undergo data processing and training stages; therefore, the quantitative calculation of their efficiency should also consider the time and space overhead of these processes. Simultaneously, the neural networks within these tools should be isolated for traditional machine learning performance analysis, and their functional roles and performance should be analyzed in relation to the fuzzing tools they involve. Furthermore, due to the black-box nature of neural networks, whether the effectiveness of these tools is truly related to the neural network, or how their performance is influenced by the neural network, remains unverified. For example, the accuracy of the trained neural network and the quantitative relationship between accuracy and the efficiency of a specific fuzzing tool; how well neural network models that perform well in image and text processing actually perform when faced with complex programming objects; and the impact of gradient information from deep learning training on subsequent mutation processes remains unexplained and unresearched.

[0028] 1. Tool Selection and Test Procedure Description For empirical analysis experiments, representative target tools should be selected, primarily information-oriented fuzzing tools based on deep learning, such as Neuzz and MTFuzz. Simultaneously, for the test program projects, software programs that are widely used in the fuzzing field, have universally applicable internal structures, and are representative should be chosen. This paper selects some software programs from the Binutils project as target test programs.

[0029] 2. Empirical Research Problem Analysis and Experimental Design 2.1 Analysis of Empirical Research Problems Compared to traditional fuzzing tools, how effective are deep learning-based fuzzing tools in improving code coverage? How do different deep learning models affect the performance of deep learning-based fuzzing tools? 1.1 Empirical Research Experimental Design The experiment used the same initial seed for all blurring tools. A 12-hour experiment was conducted for each blurring tool, and the experiment was repeated 5 times with the same configuration. The average edge coverage was then calculated for comparison.

[0030] The actual performance of a deep learning-based fuzzing tool is evaluated by comparing its edge coverage with that of a traditional fuzzing tool. Figure 3 (a) through (f) show the average results of edge coverage over 12 hours for six items including nm.

[0031] The empirical studies above reveal the following problems with existing technologies: 1. Traditional fuzzing tools often encounter imbalanced sample data in practice, and the high accuracy of the model is largely affected by the higher proportion of sample data, meaning that the neural network model falls far short of its intended purpose; 2. Traditional fuzzing tools have low information utilization rates in their neural networks. Although some feedback information from the neural network is relatively accurate, further improvements to the tool are still needed.

[0032] The first embodiment of this specification (hereinafter referred to as "Embodiment 1") provides a fuzzy testing method based on neural network gradient information guidance. The execution subject of Embodiment 1 includes, but is not limited to, a terminal, a server, an operating system, or an application; that is, the execution subject can be diverse and can be set, used, or changed as needed. Alternatively, a third-party application can assist the execution subject in executing Embodiment 1. For example, a server can execute the method in Embodiment 1, and a corresponding application can be installed on a terminal (which may be held by a user). Data transmission can occur between the terminal or application and the server, thereby assisting the server in executing the method in Embodiment 1.

[0033] The fuzz testing method based on neural network gradient information guidance provided in Example 1 includes: S101: Monitor the fuzzing speed of the program under test, and determine whether it is necessary to update the test cases by training a neural network model based on the fuzzing speed; During fuzzing of the program under test (or target program), the fuzzing speed is monitored. The fuzzing speed can be determined by the number of new branches triggered between consecutive fuzzing tests. If the number of new branches triggered in a subsequent fuzzing test decreases, the fuzzing speed can be considered to have slowed down.

[0034] The speed of fuzz testing can be used to determine whether it's necessary to update test cases by training a neural network model; that is, whether the trained neural network model is needed to update the fuzz test cases. For example, if several consecutive fuzz tests fail to trigger new branches or the number of new branches triggered is below a set value, it can be determined that it's necessary to update the test cases by training a neural network model.

[0035] S103: Determine the training dataset used to train the neural network model based on the fuzzy testing operation already performed, and use the training dataset to train the neural network model to obtain the trained target model. In Example 1, after determining that test cases need to be updated by training a neural network model, the training dataset used to train the neural network model is determined based on the fuzzing operations already performed.

[0036] Preferably, the training dataset used to train the neural network model, determined based on the fuzzing operations already performed, includes: The relationships between different branches of the tested program are determined based on the fuzzing operations already performed. Identify candidate branches that meet the rarity condition, and select branches of interest that are independent of each other from the candidate branches based on the relationship between the different branches. Test cases that can cover the branches of interest will be used as training data in the training dataset used to train the neural network model.

[0037] The following explains further how to determine the training data: Example 1, through research on neural networks, found that dependent labels can affect the training effect of neural networks. Therefore, when selecting training data, a credibility check should be performed (including verifying the ratio of positive and negative training data and verifying the training accuracy of the neural network) to filter out dependent label data. In addition to not being dependent on each other, the labels used as training data also need to possess the important characteristic of rarity.

[0038] Based on this, the relationships between different branches of the program under test can be determined from the fuzzing operations already performed. Specifically, during the instrumentation process of fuzzing, the branching status of the program under test can be recorded, and the relationships between the branches can be summarized. Since the instrumentation markers randomly generated by AFL do not contain BasicBlock information, they do not meet the requirements.

[0039] In Example 1, during the fuzzing process, MFB (Moudle + Fusion + Basic Block) was specifically used as the instrumentation identifier for the program under test. The compiler structuring technology of LLVM Pass was used for instrumentation, which enabled the recording, statistics and analysis of the relationship between different branches of the program under test.

[0040] Furthermore, during the ongoing fuzzing process, the AFL kernel-based fuzzing scheme can still be used. Within its `Fuzzing_one()` function, it's necessary to statistically analyze the rarest branches and which test cases cover them. When a rarer branch appears, the less rare branch will be deleted, thus achieving real-time updates on the rare branches.

[0041] The above content enables the analysis and statistics of the relationships between branches and whether each branch is rare.

[0042] Rarity conditions can be preset. Based on the above analysis of whether each branch is rare, branches that meet the rarity conditions can be determined and used as candidate branches. For example, branches can be sorted according to their rarity, and the rarest branches can be selected as those that meet the rarity conditions.

[0043] Based on the relationships between different branches, branches that are independent of each other are selected from the candidate branches, and these selected branches are designated as branches of interest. It is evident that the selected branches of interest satisfy both the rarity condition (scarcity) and the independence condition (independence). In practical applications, the number of branches of interest can be chosen as needed; for example, 10 branches of interest are selected from the candidate branches, meaning 10 independent branches are chosen as branches of interest.

[0044] Since branches of interest are both rare and independent of each other, test cases that can cover branches of interest also have the labels of being rare and independent of each other. Therefore, test cases that can cover branches of interest are used as training data in the training dataset used to train the neural network model.

[0045] In Example 1, the training dataset includes positive and negative training data (i.e., positive and negative samples). Positive training data refers to test cases that trigger a certain branch, while negative training data refers to test cases that prevent a certain branch from being triggered. Considering that the ratio of positive to negative training data may be unbalanced in practice, or that the number of test cases covering the aforementioned branches of interest is usually small, in addition to determining the training data in the above manner, the number of positive and negative training data in the training dataset can be further balanced and supplemented.

[0046] Preferably, balancing the number of positive and negative training data in the training dataset includes: If a test case file hits (i.e. triggers) a branch of interest in the program under test, then a mutation operation is performed on the test case file, including: mutating bytes at various positions in the test case file. Specifically, each time, based on the test case file, bytes at a specific position in the test case file are mutated, and the position of the mutated byte is different each time.

[0047] After each mutation, it is determined whether the resulting test case file hits the branch of interest to identify key positions. The key position is the location associated with that branch of interest. Specifically, if the test case file hits a branch of interest, and the test case file resulting from mutating a byte at a certain position also hits that branch of interest, then that position is associated with that branch of interest (called the key position), and that byte is the key byte of that branch of interest. In this way, by monitoring the running status of the program under test in the mutated test case file, key positions and key bytes are determined, achieving the goal of analyzing the relationship between bytes and branches.

[0048] Once all bytes at each position in the test case file have been mutated, all critical positions and key bytes in the test case file are identified. The bytes at non-critical positions in the test case file are then mutated, and the resulting new test case file is used as training data in the training dataset.

[0049] Furthermore, for a given branch of interest, if the test case file generated after each mutation also hits that branch of interest, then another mutation operation can be performed on the test case file generated after that mutation. In other words, regardless of whether the test case file is the original one or obtained through mutation, as long as it hits the branch of interest, a mutation operation can be performed on it; therefore, the mutation operation can theoretically be performed indefinitely. Of course, the termination or cancellation of the mutation operation can be controlled, for example, terminating or stopping the mutation when the positive and negative training data in the training dataset reach a balance.

[0050] The balancing operation described above generates new training data while ensuring that rare branches are hit, resulting in a more balanced number of positive and negative training data points, thus achieving the goal of adjusting the balance level. In practical applications, after balancing, the ratio of positive to negative training data in the training dataset remains approximately 1:1.

[0051] In Example 1, after obtaining the training dataset, the training data in the training dataset can be used to train the neural network model (the neural network model can be a five-layer fully connected neural network), and the trained model is called the target model.

[0052] S105: Use the target model to predict the test case file to be processed and generate gradient information; In Example 1, the target model can be used to predict the test case file to be processed, generate gradient information, and the gradient information can be mapped to the bytes of the test case file to be processed.

[0053] S107: For any type of test case file to be processed, determine the processing strategy corresponding to that type of test case file, and process the test case file to be processed using the gradient information according to the processing strategy to obtain the target test case file; In Example 1, the test case files to be processed can include various types, such as test case files that did not participate in neural network training or test case files that hit the aforementioned branches of interest. For different types of test case files, corresponding processing strategies can be determined, and then the test case files to be processed can be processed accordingly. The following explanation uses test case files that did not participate in neural network training or test case files that hit the aforementioned branches of interest as examples to illustrate different cases: Scenario 1 For test case files that did not participate in neural network training, the corresponding processing strategy includes: mutating the bytes in the test case files that did not participate in neural network training whose predicted label is a specific value (hereinafter, the specific value is 0).

[0054] Specifically, test case files that have not participated in neural network training are predicted by the target model. The higher gradient information fed back from these predictions corresponds to more important bytes that result in a predicted label of 0. These more important bytes are mutated to increase the likelihood that the mutated test case files will hit branches of interest. Furthermore, in the process of hitting branches of interest, test case files that hit branches that have not appeared before may be generated.

[0055] Scenario 2 For test case files that hit the aforementioned branch of interest, the corresponding processing strategy includes: protecting the bytes in the test case files that did not participate in neural network training through the gradient information, which caused the branch of interest to be hit, and mutating other bytes.

[0056] Specifically, when a test case file hits a branch of interest, gradient information can be used to protect the bytes in the test case file that caused the branch to be hit, while other bytes in the test case file are mutated. This ensures that if the branch of interest continues to be hit, other rarer branches under that branch of interest have a greater probability of being hit.

[0057] In Example 1, mutating bytes can include: performing integer addition or subtraction and / or bitwise flipping and / or random mutation on bytes. That is, regardless of the type of test case file to be processed, byte mutation can be performed using the following strategies (hereinafter, the bytes to be mutated can be uniformly referred to as position-determined bytes): Integer addition / subtraction: Increment or subtract one from all bytes at a defined position. Each test case file to be processed can generate two mutated files (i.e., new test case files) using this mutation method.

[0058] Bitwise flipping: This method flips bytes at specific positions in the test case file at intervals of space "S", i.e., 1 becomes 0, and 0 becomes 1. The number of mutated files generated varies depending on the value of "S". For example, if "S" is 2, 4, or 8, each test case file can generate 3 mutated files using this method.

[0059] Random mutation: For bytes at a fixed location, several locations are randomly selected for random data modification mutation. In practical applications, the number of selected locations can be controlled, thereby controlling the number of mutated files generated. For example, each test case file to be processed can generate 10 or more mutated files using this mutation method.

[0060] In Example 1, after predicting and obtaining gradient information from the test case file to be processed, the gradient information ranking can be delivered as feedback information to the mutation function, which then performs the aforementioned mutation. Due to the special mutation strategy adopted in Example 1, when ranking the gradient information, it is not necessary to statistically determine the positive or negative value; only the gradient data needs to be ranked numerically.

[0061] In Example 1, each type of test case file is processed according to its corresponding processing strategy, and the resulting mutation file can be used as the target test case file.

[0062] S109: Perform fuzz testing on the program under test using the target test case file.

[0063] Once the target test case file is obtained, it can be used to perform fuzz testing on the program under test and obtain the fuzz test results.

[0064] In addition, corresponding fuzzing tools can be developed to execute Example 1; that is, the execution entity of Example 1 can be a fuzzing tool. One such fuzzing tool is named S-Fuzzer, and Example 1 will be further explained below based on S-Fuzzer.

[0065] refer to Figure 4 and Figure 5The neural network inside S-Fuzzer learns the relationship between the input test case file (i.e. the test case file to be processed) and the branch behavior of the program under test. By instrumenting the program under test with different mutation strategies, it can cover more internal branch paths faster and ultimately discover internal vulnerabilities in the program under test.

[0066] During test case execution, S-Fuzzer records, checks the credibility of, and filters rare branches and test cases that trigger rare branches, continuously updating information on branches of interest. While running traditional fuzzing tools, S-Fuzzer also starts a runtime monitoring module. When monitoring detects difficulty in finding new program execution paths (i.e., fuzzing speed slows down), it runs a neural network training module to train the neural network. New test cases (i.e., target test case files) are generated by mutating the gradient information of the trained target model, and these new test cases are then used as initial seeds for fuzzing.

[0067] The following is an example of the internal algorithm of S-Fuzzer:

[0068] As can be seen from the internal algorithm described above, the speed of fuzzing is monitored (line 5) to determine whether it is necessary to update the test case file (line 10) by training the neural network (line 9). The data used for neural network training is collected from the normal fuzzing process, which requires relevant branch coverage information (InforBranch in line 6).

[0069] The design principle of the S-Fuzzer tool is that when the fuzzing tool under the traditional genetic algorithm runs slowly, it uses the feedback gradient information of the deep learning neural network to mutate the existing test cases into new test cases that may be covered by new branches. Then, it updates the test case set (i.e. the target test case file) and performs fuzzing. In this way, the gradient information solves the problem of fuzzing getting stuck in certain branches and unable to break through.

[0070] When balancing and supplementing the training dataset, FairFuzz's lightweight dynamic analysis techniques can be used.

[0071] In Example 1, the performance of S-Fuzzer, AFL, and Neuzz was compared under the Binutils project. Data from S-Fuzzer, its control group Neuzz, and AFL were collected after twelve hours of operation. S-Fuzzer was run five times with the same configuration, and the average of the experimental results was taken. The results are shown in Table 1 below:

[0072] Table 1 As shown in Table 1, S-Fuzzer's edge branch coverage is significantly better than AFL's, and in most projects, S-Fuzzer also outperforms Neuzz. This demonstrates that S-Fuzzer's operation improves the accuracy of neural network gradient information, allowing gradient feedback to play a truly guiding role.

[0073] Example 1 can achieve the following beneficial effects: In Example 1, during fuzzing, test cases for rare branches and their corresponding covering branches are recorded, their reliability checked, and filtered. Information about the rarest and most independent branches is continuously updated. When monitoring reveals that it is difficult to find a new program execution path, neural network training is initiated. New test cases are generated through gradient information mutation within the neural network. These new test cases are then used as initial seeds for fuzzing, effectively improving the fuzzing results and efficiency.

[0074] Example 1, based on the training of the fuzz test seed, further analyzes the performance of the neural network and reverse-engineers the high-priority mutated byte bits to further improve the efficiency of fuzz testing.

[0075] Example 1 further optimizes the data processing method and the acquisition of post-trained model information. Based on the model information interpreted by the neural network, more valuable byte data is mutated to improve fuzz testing efficiency.

[0076] Example 1 focuses more on neural network interpretation and training data optimization, which can improve fuzz testing efficiency.

[0077] Example 1 can be used for software development testing, achieving high-efficiency and high-quality software testing results.

[0078] The second embodiment of this specification provides a fuzz testing device based on neural network gradient information guidance, corresponding to the first embodiment. The device includes: The runtime status monitoring module is used to monitor the fuzzing speed of the program under test and determine whether it is necessary to update the test cases by training a neural network model based on the fuzzing speed. The test update module is used to determine the training dataset used to train the neural network model based on the fuzz test operations that have been performed, and to train the neural network model using the training dataset to obtain the trained target model. Furthermore, the target model is used to predict and generate gradient information for the test case files to be processed; Furthermore, for any type of test case file to be processed, determine the processing strategy corresponding to that type of test case file, and process the test case file to be processed using the gradient information according to the processing strategy to obtain the target test case file; And, fuzz testing is performed on the program under test using the target test case file; The training dataset used to train the neural network model, determined based on the fuzzing operations already performed, includes: The relationships between different branches of the tested program are determined based on the fuzzing operations already performed. Identify candidate branches that meet the rarity condition, and select branches of interest that are independent of each other from the candidate branches based on the relationship between the different branches. Test cases that can cover the branches of interest will be used as training data in the training dataset used to train the neural network model.

[0079] Preferably, the test case files to be processed include test case files that did not participate in neural network training, and the processing strategy for determining the test case files that did not participate in neural network training includes: In test case files that did not participate in neural network training, the bytes with predicted labels of specific values ​​are mutated.

[0080] Preferably, the feature value is 0.

[0081] Preferably, the test case files to be processed include test case files that hit the branch of interest, and the processing strategy for determining the test case files that did not participate in neural network training includes: The gradient information is used to protect the bytes in the test case files that did not participate in the neural network training, causing the branch of interest to be hit, and mutating other bytes.

[0082] Preferably, byte mutation includes: Perform integer addition, subtraction, bitwise flipping, and / or random mutation on bytes.

[0083] Preferably, before training the neural network model using the training dataset, the method further includes: The number of positive and negative training data in the training dataset is balanced.

[0084] Preferably, balancing the number of positive and negative training data in the training dataset includes: If a test case file hits a branch of interest in the program under test, then the bytes at each position in the test case file are mutated respectively; After each mutation, it is determined whether the mutated test case file can hit the branch of interest in order to determine the key position; wherein, the key position is the position associated with the branch of interest. The bytes in the non-critical positions of the test case file are mutated, and the resulting new test case file is used as training data in the training dataset.

[0085] Preferably, the test update module is also used to: use MFB as an instrumentation identifier.

[0086] Preferably, the test update module is also used for instrumentation using LLVM's Pass compiler structuring technology.

[0087] The neural network training module in Example 1 can be part of the aforementioned test and update module.

[0088] The contents not described in detail in Embodiment 1 and Embodiment 2 can be referred to each other. Embodiment 2 can achieve the same beneficial effects as Embodiment 1. The above embodiments can be used in combination.

[0089] The above description is merely an embodiment of this specification and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A fuzzy testing method based on neural network gradient information guidance, characterized in that, The method includes: Monitor the fuzzing speed of the program under test, and determine whether it is necessary to update the test cases by training a neural network model based on the fuzzing speed; The training dataset used to train the neural network model is determined based on the fuzzy testing operations already performed. The neural network model is then trained using the training dataset to obtain the trained target model. The target model is used to predict and generate gradient information for the test case files to be processed. For any type of test case file to be processed, determine the processing strategy corresponding to that type of test case file, and process the test case file of that type using the gradient information according to the processing strategy to obtain the target test case file; The program under test is fuzz-tested using the target test case file. The training dataset used to train the neural network model, determined based on the fuzzing operations already performed, includes: The relationships between different branches of the tested program are determined based on the fuzzing operations already performed. Identify candidate branches that meet the rarity condition, and select branches of interest that are independent of each other from the candidate branches based on the relationship between the different branches. Test cases that can cover the branches of interest will be used as training data in the training dataset used to train the neural network model.

2. The method as described in claim 1, characterized in that, The test case files to be processed include test case files that did not participate in neural network training. The processing strategies for determining the test case files that did not participate in neural network training include: In test case files that did not participate in neural network training, the bytes with predicted labels of specific values ​​are mutated.

3. The method as described in claim 2, characterized in that, The characteristic value is 0.

4. The method as described in claim 1, characterized in that, The test case files to be processed include test case files that hit the branch of interest. The processing strategies for test case files that did not participate in neural network training include: The gradient information is used to protect the bytes in the test case files that did not participate in the neural network training, causing the branch of interest to be hit, and mutating other bytes.

5. The method according to any one of claims 2 to 4, characterized in that, Mutating bytes includes: Perform integer addition, subtraction, bitwise flipping, and / or random mutation on bytes.

6. The method as described in claim 1, characterized in that, Before training the neural network model using the training dataset, the method further includes: The number of positive and negative training data in the training dataset is balanced.

7. The method as described in claim 6, characterized in that, Balancing the number of positive and negative training data in the training dataset includes: If a test case file hits a branch of interest in the program under test, then the bytes at each position in the test case file are mutated respectively; After each mutation, it is determined whether the mutated test case file can hit the branch of interest in order to determine the key position; wherein, the key position is the position associated with the branch of interest. The bytes in the non-critical positions of the test case file are mutated, and the resulting new test case file is used as training data in the training dataset.

8. The method as described in claim 1, characterized in that, The method further includes: Use MFB as the marker for the stake.

9. The method as described in claim 1 or 8, characterized in that, The method further includes: Instrumentation is performed using LLVM's Pass compiler structuring techniques.

10. A fuzzy testing device based on neural network gradient information guidance, characterized in that, The device includes: The runtime status monitoring module is used to monitor the fuzzing speed of the program under test and determine whether it is necessary to update the test cases by training a neural network model based on the fuzzing speed. The test update module is used to determine the training dataset used to train the neural network model based on the fuzz test operations that have been performed, and to train the neural network model using the training dataset to obtain the trained target model. Furthermore, the target model is used to predict and generate gradient information for the test case files to be processed; Furthermore, for any type of test case file to be processed, determine the processing strategy corresponding to that type of test case file, and process the test case file to be processed using the gradient information according to the processing strategy to obtain the target test case file; And, fuzz testing is performed on the program under test using the target test case file; The training dataset used to train the neural network model, determined based on the fuzzing operations already performed, includes: The relationships between different branches of the tested program are determined based on the fuzzing operations already performed. Identify candidate branches that meet the rarity condition, and select branches of interest that are independent of each other from the candidate branches based on the relationship between the different branches. Test cases that can cover the branches of interest will be used as training data in the training dataset used to train the neural network model.

Images