Deep learning based fuzzing test case construction method
By learning the features of fuzzy test cases and filtering similar samples through the self-attention generative adversarial network SAGAN, the low generation efficiency and similarity problems in the existing technology are solved, and the efficient generation of diverse test cases is achieved, thus improving the efficiency of fuzz testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2023-04-10
- Publication Date
- 2026-04-24
AI Technical Summary
In existing fuzzing methods, generative adversarial networks cannot effectively handle long-distance dependencies, resulting in low test case generation efficiency and the generation of similar test cases, which increases computational overhead.
We employ a self-attention generative adversarial network (SAGAN) to learn fuzzy test case features. By using the SAGAN network to learn test case features and editing distance to filter similar samples, we improve the diversity and generation efficiency of test cases.
It improves the efficiency of fuzz testing, ensures the quality and diversity of generated test cases, reduces computational overhead, and increases testing efficiency.
Smart Images
Figure CN116225964B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of software testing technology, and further relates to a fuzzy test case construction method, specifically a deep learning-based fuzzy test case construction method that can be used for fuzzy testing. Background Technology
[0002] Fuzzing is an automated method for detecting application vulnerabilities. It generates a large number of test cases, inputs them into the target program, and detects abnormal behavior during program execution to uncover vulnerabilities. Fuzzing is not limited by the internal implementation details and complexity of the program under test and has good reusability, making it one of the most successful techniques for detecting program vulnerabilities. Based on the method of test case generation, fuzzing can be divided into two categories: generation-based fuzzing and mutation-based fuzzing. Generation-based fuzzing learns the syntax of the program input to generate fuzzy test cases. Mutation-based fuzzing obtains test cases by mutating the initial seed file. However, generation-based fuzzing requires manually obtaining the syntax of the program input, which is obviously difficult and inefficient. Mutation-based fuzzing only requires a valid initial seed file to generate a large number of test cases.
[0003] The initial seed file is a crucial factor affecting the performance of mutation-based fuzzing. The purpose of fuzzing is to search for test cases that might cause abnormal program behavior, such as crashes. This is done by generating test case inputs from the mutation seed file and executing them to explore the program's adjacent input space, thereby finding vulnerabilities. However, fuzzing faces several challenges: the target program is often very complex, with a nearly infinite input space; and due to the uncertainty of the mutation strategy, most test cases generated from the mutation seed file cannot pass parsing and are rejected by the program. Without a high-quality seed file, generating test cases that can pass parsing and explore the program state based on mutation is highly inefficient. Therefore, generating high-quality inputs to explore the program state is a challenge for mutation-based fuzzing and a difficult problem to improve its efficiency.
[0004] Existing technologies have begun to utilize Generative Adversarial Networks (GANs) to generate test cases. Leveraging their powerful learning capabilities, GAN-based fuzzing methods use prior knowledge to learn the probability distribution of test cases and generate high-quality test cases. However, existing methods cannot handle long-range dependencies, limiting the representational power of GANs and thus reducing the efficiency of test case generation. In addition, although GANs can generate a large number of inputs, the problem of input similarity arises. If similar inputs are not excluded and fuzzing is performed directly, it will cause additional computational overhead, resulting in low fuzzing efficiency. Summary of the Invention
[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a deep learning-based method for constructing fuzzy test cases. This method utilizes Self-Attention Generative Adversarial Networks (SAGAN) to solve the problems of low efficiency and duplicate samples in existing test case acquisition methods. The method leverages SAGAN to learn the features of existing fuzzy test cases, thereby rapidly generating a large number of test cases. Furthermore, it uses edit distance to calculate the similarity of test cases, filtering out those exceeding a set threshold. This invention can obtain high-quality test cases and effectively improve fuzzing efficiency.
[0006] The idea behind implementing the method of this invention is as follows: First, collect existing test cases as a dataset, then preprocess the data to meet the input requirements of SAGAN, then use SAGAN to train and obtain samples, filter out the generated duplicate samples, and finally use the generated samples as test cases for fuzz testing.
[0007] To achieve the above objectives, the technical solution of the present invention includes:
[0008] (1) Collect files of different formats from the Internet as input for fuzzing, perform fuzzing on the application to obtain the first test case, and use the test case to build a dataset;
[0009] (2) Preprocess the dataset:
[0010] (2.1) Use the first test case in the binary read dataset to obtain a binary byte stream;
[0011] (2.2) The binary byte stream is encoded using the Base64 binary encoding method to obtain an encoded string consisting of 64 characters and "=". The encoded string is then converted into decimal numbers to obtain the preprocessed dataset.
[0012] (3) Construct a generative adversarial network model SAGAN based on self-attention mechanism, including constructing generator and discriminator structures, and using the value function of Wasserstein generative adversarial network WGAN to obtain a generative adversarial network model SAGAN based on self-attention mechanism.
[0013] (4) Train the constructed SAGAN network model:
[0014] (4.1) Randomly select m data samples from the preprocessed dataset to form a data sample set {x}. 1 ,x 2 ,…,x i,…,x m}, i = 1, 2, ... m;
[0015] (4.2) Keep the generator parameters unchanged and train the discriminator: Construct the first random sample set {z1} containing m elements. 1 ,z1 2 ,…,z1 m}, take the i-th random sample z1 i The input is fed into the generator to obtain the first output result set {y1}. 1 ,y1 2 ,…,y1 m}; the i-th output result y1 in the set i With the i-th data sample x i The data is fed into the discriminator, the discriminator loss function is calculated, then the discriminator loss function is backpropagated, and the discriminator parameters w are updated using the Adam optimization algorithm.
[0016] (4.3) Keep the discriminator parameters unchanged and train the generator: Construct a second random sample set {z2} containing m elements. 1 z2 2 ,…,z2 m}, and take the i-th random sample z2 i The input is fed into the generator to obtain the second output result set {y2}. 1 ,y2 2 ,…,y2 m}, and the i-th output result y2 in this set i The input is fed into the discriminator, the generator's loss function is calculated, then the generator's loss function is backpropagated, and the generator parameters θ are updated using the Adam optimization algorithm.
[0017] (4.4) Repeat (4.1) and (4.3) until the generator parameters θ converge, then the SAGAN training is complete and the trained SAGAN network model is obtained.
[0018] (5) Use the trained SAGAN network model to obtain the final test cases for fuzz testing:
[0019] (5.1) Generate samples using the trained SAGAN model and transform them to obtain the second set of test cases;
[0020] (5.2) The similarity between all test cases in the second test case set is calculated using edit distance, and the calculation results are compared with a pre-set similarity threshold. If the similarity is greater than the threshold, the two test cases are determined to be similar and one of them is filtered out; otherwise, no filtering operation is performed.
[0021] (5.3) Obtain the final set of test cases for fuzz testing.
[0022] Compared with the prior art, the present invention has the following advantages:
[0023] First, because the present invention uses a self-attention mechanism to handle long-distance dependencies when constructing the generative adversarial network SAGAN, the representation ability of SAGAN is improved, thereby enabling the rapid generation of a large number of test cases.
[0024] Secondly, after generating samples using the trained SAGAN model, this invention employs an edit distance algorithm to process similar inputs in the sample set, ensuring that there are no similar samples in the final test cases, thus guaranteeing the diversity of test cases and effectively improving the efficiency of fuzz testing. Attached Figure Description
[0025] Figure 1 This is a schematic diagram of the data preprocessing process in this invention;
[0026] Figure 2 This is a schematic diagram of the self-attention mechanism structure in this invention;
[0027] Figure 3 This is a schematic diagram of the SAGAN generator and discriminator in this invention; where (a) represents the schematic diagram of the generator and (b) represents the schematic diagram of the discriminator.
[0028] Figure 4 This is a flowchart illustrating the implementation of the method of the present invention;
[0029] Figure 5 The simulation results of constructing test cases and randomly selecting test cases for this invention are shown in the figure; where (a) represents the simulation results of fuzz testing on cjpeg, and (b) represents the simulation results of fuzz testing on infotocap. Detailed Implementation
[0030] The present invention will now be further described with reference to the accompanying drawings.
[0031] Example 1: Refer to Appendix Figure 4 The present invention proposes a method for constructing fuzzy test cases based on deep learning, which specifically includes the following steps:
[0032] Step 1: Collect files of different formats from the internet as input for fuzz testing, perform fuzz testing on the application to obtain the first test case, and use the test case to build a dataset.
[0033] Because fuzzing can quickly generate a large number of test cases, and some test cases can reach deeper program paths or even crash the program, the test cases generated by fuzzing can be used as a dataset. Using regular files collected from the internet as input to fuzz the application, the resulting test cases are used as the dataset. To ensure the diversity of test cases in the dataset, this embodiment considers collecting files of different formats.
[0034] Step 2, refer to Figure 1 Preprocess the dataset:
[0035] (2.1) Use the first test case in the binary read dataset to obtain a binary byte stream.
[0036] Since the test cases in the dataset have different types and sizes, normal reading methods may not work. Considering that any file can be read in binary, this invention uses binary reading in this step to obtain a binary string.
[0037] (2.2) The binary byte stream is encoded using the Base64 binary encoding method to obtain an encoded string composed of 64 characters and "=". The encoded string is then converted into decimal numbers to obtain the preprocessed dataset, which is to convert the data into a matrix.
[0038] Step 3: Construct a SAGAN network model based on a self-attention mechanism. This includes constructing the generator and discriminator structures and using the value function of the Wasserstein Generative Adversarial Network (WGAN) to obtain the SAGAN network model based on a self-attention mechanism. Specifically, the SAGAN network model is constructed using the deep learning library PyTorch.
[0039] In this step, the invention utilizes the PyTorch deep learning library to build the SAGAN structure: (Refer to...) Figure 3 Construct the generator and discriminator structures for SAGAN, referring to... Figure 2 A self-attention module is built for use by the generator and discriminator. To address training instability, the value function of the Generative Adversarial Network (WGAN) using Wasserstein is employed, and a spectral normalization module is constructed to satisfy the 1-Lipschitz constraint. Finally, the SAGAN network structure is completed.
[0040] Reference Figure 3In (a) of the above, the generator is constructed as follows: two convolutional layers are used to extract shallow features, and the parameters of the convolutional layers are set using the spectral normalization method. The ReLU function is used as the activation function. Then, two convolutional layers are used to extract deep features, and a self-attention mechanism is added. Finally, a convolutional layer is added, and the Tanh function is selected as the activation function.
[0041] Reference Figure 3 In (b) of the diagram, the discriminator is constructed by using four convolutional layers and setting the parameters of the convolutional layers using spectral normalization. The LeakyRelu function is selected as the activation function, and a self-attention mechanism is added to the next two convolutional layers. Finally, a convolutional layer is added.
[0042] Step 4: Train the constructed SAGAN network model:
[0043] (4.1) Randomly select m data samples from the preprocessed dataset to form a data sample set {x}. 1 ,x 2 ,…,x i ,…,x m}, i = 1, 2, ... m;
[0044] (4.2) Keep the generator parameters unchanged and train the discriminator: Construct the first random sample set {z1} containing m elements. 1 ,z1 2 ,…,z1 m}, take the i-th random sample z1 i The input is fed into the generator to obtain the first output result set {y1}. 1 ,y1 2 ,…,y1 m}; the i-th output result y1 in the set i With the i-th data sample x i The data is fed into the discriminator, the discriminator loss function is calculated, then the discriminator loss function is backpropagated, and the discriminator parameters w are updated using the Adam optimization algorithm.
[0045] The process of updating the discriminator parameter w using the Adam optimization algorithm is as follows:
[0046]
[0047] w←w+α·Adam(w,g w ),
[0048] Among them, g w This represents the value of the discriminator loss function after backpropagation. f represents the gradient of the discriminator parameter w. w G represents the discriminator. θ Let α represent the generator and α represent the learning rate.
[0049] (4.3) Keep the discriminator parameters unchanged and train the generator: Construct a second random sample set {z2} containing m elements. 1 z2 2 ,…,z2 m}, and take the i-th random sample z2 i The input is fed into the generator to obtain the second output result set {y2}. 1 ,y2 2 ,…,y2 m}, and the i-th output result y2 in this set i The input is fed into the discriminator, the generator's loss function is calculated, then the generator's loss function is backpropagated, and the generator parameters θ are updated using the Adam optimization algorithm.
[0050] The process of updating the generator parameter θ using the Adam optimization algorithm is as follows:
[0051]
[0052] θ←θ-α·Adam(θ,g θ )
[0053] Among them, g θ This represents the value of the generator loss function after backpropagation. f represents the gradient of the generator parameter θ; w G represents the discriminator. θ Let α represent the generator and α represent the learning rate.
[0054] (4.4) Repeat (4.1) and (4.3) until the generator parameters θ converge, then the SAGAN training is complete and the trained SAGAN network model is obtained.
[0055] Step 5: Use the trained SAGAN network model to obtain the final test cases for fuzz testing:
[0056] (5.1) Generate samples using the trained SAGAN model and transform them to obtain the second test case set. Since the samples generated by SAGAN are in matrix form, they need to be transformed. This transformation is the inverse operation of step 2. After the transformation is completed, the input file, i.e., the second test case set, is obtained.
[0057] (5.2) To solve the problem of test case duplication, edit distance is used to calculate the pairwise similarity between all test cases in the second test case set, and the calculation results are compared with a pre-set similarity threshold. If the result is greater than the threshold, the two test cases are determined to be similar and one of them is filtered out; otherwise, no filtering operation is performed.
[0058] The similarity between all test cases in the second test case set is calculated using edit distance, where the edit distance s between the i-th test case and the j-th test case is... i,j Calculate using the following formula:
[0059]
[0060] Among them, edit(str i str j ) indicates that the string str i Convert to str j The operand, len(str) i ) represents the string str i The length of str j ) represents the string str j The length of the string, max(len(str) i ),len(str j )) means len(str i ) and len(str j The maximum value of ).
[0061] In this embodiment, a similarity threshold of 0.9 is set. Therefore, when the calculated similarity s between two test cases is... i,j If the value is greater than 0.9, the two test cases are considered similar, and one of them is filtered out to ensure the diversity of test cases.
[0062] (5.3) Obtain the final set of test cases for fuzz testing.
[0063] Using the filtered test cases from this invention, i.e., the final test cases, to perform fuzz testing yields results comparable to those obtained by directly selecting test cases for fuzz testing. Figure 5 As shown in the figure, it can be seen that the effect of the present invention is significantly better than that of the existing methods.
[0064] The parts of this invention not described in detail are common knowledge to those skilled in the art.
[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art, after understanding the content and principle of the present invention, may make various modifications and changes in form and detail without departing from the principle and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.
Claims
1. A method for constructing fuzzy test cases based on deep learning, characterized in that, Includes the following steps: (1) Collect files of different formats from the Internet as input for fuzzing, perform fuzzing on the application to obtain the first test case, and use the test case to build a dataset; (2) Preprocess the dataset: (2.1) Use the first test case in the binary read dataset to obtain a binary byte stream; (2.2) The binary byte stream is encoded using the Base64 binary encoding method to obtain an encoded string consisting of 64 characters and "=". The encoded string is then converted into decimal numbers to obtain the preprocessed dataset. (3) Construct a generative adversarial network model SAGAN based on self-attention mechanism, including constructing generator and discriminator structures, and using the value function of Wasserstein generative adversarial network WGAN to obtain a generative adversarial network model SAGAN based on self-attention mechanism. The generator is constructed as follows: two convolutional layers are used to extract shallow features, and the parameters of the convolutional layers are set using the spectral normalization method. The ReLU function is used as the activation function. Then, two convolutional layers are used to extract deep features, and a self-attention mechanism is added. Finally, a convolutional layer is added, and the Tanh function is selected as the activation function. The discriminator is constructed by: using four convolutional layers, setting the parameters of the convolutional layers using spectral normalization, selecting LeakyReLU as the activation function, adding a self-attention mechanism in the next two convolutional layers, and finally adding a convolutional layer. (4) Train the constructed SAGAN network model: (4.1) Randomly select m data samples from the preprocessed dataset to form a data sample set {x}. 1 ,x 2 ,…,x i , …,x m }, i=1,2,...m; (4.2) Keep the generator parameters unchanged and train the discriminator: Construct the first random sample set {z1} containing m elements. 1 ,z1 2 ,…,z1 m }, take the i-th random sample z1 i The input is fed into the generator to obtain the first output result set {y1}. 1 ,y1 2 ,…,y1 m }; the i-th output result y1 in the set i With the i-th data sample x i The data is fed into the discriminator, the discriminator loss function is calculated, then the discriminator loss function is backpropagated, and the discriminator parameters w are updated using the Adam optimization algorithm. (4.3) Keep the discriminator parameters unchanged and train the generator: Construct a second random sample set {z2} containing m elements. 1 z2 2 ,…,z2 m }, and take the i-th random sample z2 i The input is fed into the generator to obtain the second output result set {y2}. 1 ,y2 2 ,…,y2 m }, and the i-th output result y2 in this set i The input is fed into the discriminator, the generator's loss function is calculated, then the generator's loss function is backpropagated, and the generator parameters θ are updated using the Adam optimization algorithm. (4.4) Repeat (4.1) and (4.3) until the generator parameters θ converge, then the SAGAN training is complete and the trained SAGAN network model is obtained; (5) Use the trained SAGAN network model to obtain the final test cases for fuzz testing: (5.1) Generate samples using the trained SAGAN model and transform them to obtain the second test case set; (5.2) The similarity between all test cases in the second test case set is calculated using edit distance, and the calculation result is compared with a pre-set similarity threshold. If the similarity is greater than the threshold, the two test cases are judged to be similar and one of them is filtered out; otherwise, no filtering operation is performed. The similarity between all test cases in the second test case set is calculated using edit distance. The test case and the first Edit distance between test cases Calculate using the following formula: in, Indicates the string Convert to operands, Represents a string Length, Represents a string Length, express and The maximum value; (5.3) Obtain the final set of test cases for fuzz testing.
2. The method according to claim 1, characterized in that: Step (3) describes the construction of a generative adversarial network (SAGAN) model based on a self-attention mechanism, which is specifically built using the deep learning library PyTorch.
3. The method according to claim 1, characterized in that: Step (4.2) involves updating the discriminator parameters w using the Adam optimization algorithm. The update process is as follows: ， ， in, This represents the value of the discriminator loss function after backpropagation. This represents the gradient of the discriminator parameter w. Indicates the discriminator, Represents a generator. This represents the learning rate.
4. The method according to claim 1, characterized in that: Step (4.3) involves updating the generator parameters using the Adam optimization algorithm. The update process is as follows: in, This represents the value of the generator loss function after backpropagation. Indicate generator parameters The gradient; Indicates the discriminator, Represents a generator. This represents the learning rate.