Test method, device, electronic equipment and readable medium of application program

Abstract

This invention provides a method, apparatus, electronic device, and readable medium for testing applications, relating to the field of computer technology. In this method, in a scenario with fixed memory addresses, variant test cases of the program under test are obtained, and the program under test is tested based on these variant test cases to obtain a first test result. Variant test cases whose abnormal output is characterized by the first test result are identified as candidate test cases. In a scenario with randomized memory address layout, the program under test is tested based on the candidate test cases to obtain a second test result. Based on the second test result, target test cases are selected from the candidate test cases and output; these target test cases characterize security vulnerabilities in the program under test. This avoids omitting variant test cases that could trigger vulnerabilities in the output test cases, ensuring comprehensive testing and improving test reliability.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and readable medium for testing applications. Background Technology

[0002] With the continuous development of computer technology, there are more and more applications available. Before deploying an application, it needs to be tested to identify any security vulnerabilities, thereby optimizing the application and ensuring its security.

[0003] In related technologies, default mutation test cases are often used to test the program under test directly in scenarios with randomized memory address layout. These mutation test cases include unexpected inputs. If the test result indicates abnormal output, it means that the input in the mutation test case causes the program under test to run abnormally, and the program under test has a security vulnerability. However, this approach suffers from incomplete testing and low test reliability. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and readable medium for testing applications, which can solve the problems of incomplete testing and low testing reliability.

[0005] To address the above problems, embodiments of the present invention disclose a method for testing applications, the method comprising: In a scenario with fixed memory addresses, obtain the variant test cases of the program under test, and test the program under test based on the variant test cases to obtain the first test result; The variant test cases whose first test results represent abnormal outputs are identified as candidate test cases. In a scenario where memory address layout is randomized, the program under test is tested based on the candidate test cases to obtain a second test result; Based on the second test result, the candidate test cases are selected and the target test cases are output; the target test cases are used to characterize the security vulnerabilities existing in the program under test.

[0006] On the other hand, embodiments of the present invention disclose an application testing apparatus, the apparatus comprising: The first testing module is used to obtain variant test cases of the program under test in a scenario with fixed memory address, and to test the program under test based on the variant test cases to obtain the first test result. The first determining module is used to determine the variant test cases whose abnormal output is characterized by the first test result as candidate test cases; The second testing module is used to test the program under test based on the candidate test cases in a scenario of randomized memory address layout, and obtain the second test result. The first output module is used to filter from the candidate test cases based on the second test result and output the target test cases; the target test cases are used to characterize the security vulnerabilities existing in the program under test.

[0007] In another aspect, embodiments of the present invention disclose an electronic device, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store executable instructions, which cause the processor to execute the aforementioned method.

[0008] This invention also discloses a machine-readable medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described above.

[0009] The embodiments of this invention offer the following advantages: The application testing method provided by these embodiments, in scenarios with fixed memory addresses, obtains variant test cases for the program under test, and tests the program under test based on these variant test cases to obtain a first test result. Variant test cases whose abnormal output is represented by the first test result are identified as candidate test cases. In scenarios with randomized memory address layouts, the program under test is tested based on the candidate test cases to obtain a second test result. Based on the second test result, target test cases are selected from the candidate test cases and output; these target test cases are used to characterize security vulnerabilities in the program under test. Thus, the application under test is first tested in scenarios with fixed memory addresses, and variant test cases that trigger vulnerabilities when the memory layout remains unchanged are identified as candidate test cases. Further testing is then conducted in scenarios with randomly changing memory address layouts, and test cases that also trigger vulnerabilities in actual use scenarios are selected from the candidate test cases and output as target test cases. This avoids omitting variant test cases that trigger vulnerabilities in the output test cases, ensuring comprehensive testing and improving testing reliability.

[0010] Furthermore, by avoiding including test cases in the output that would not trigger vulnerabilities in actual use scenarios, the accuracy of the output results is ensured. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below.

[0012] Figure 1This is a flowchart of the steps of a testing method for an application provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a processing flow provided in an embodiment of the present invention; Figure 3 This is a block diagram of an application testing device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0014] First, let's describe an application scenario related to this invention. In a fuzzing scenario, a large number of unexpected inputs are provided to the application to be tested (hereinafter referred to as the program under test), and the program under test is monitored for abnormal outputs, such as crash signals output when assertion failures or memory errors occur. Unexpected inputs are input data that do not conform to the expected format or range of the program under test. In fuzzing, unexpected inputs are provided to the program under test to trigger abnormal behavior, thereby discovering potential vulnerabilities.

[0015] In related technologies, multiple mutation test cases are often used to test the program under test (DUT) and monitor for abnormal output. If an unexpected input from a mutation test case is provided to the DUT, causing abnormal output during execution, the mutation test case is considered to have triggered a vulnerability. This mutation test case can then be provided to technical personnel for evaluation and optimization. However, related technologies directly test in scenarios with randomized memory address layouts. In such scenarios, the load base addresses of memory regions such as code segments, heaps, and stacks are randomized each time the program starts. Therefore, it is difficult to trigger vulnerabilities that require precise memory layout (e.g., certain types of information leakage or complex vulnerabilities such as Use-After-Free, UAF). Consequently, mutation test cases that would trigger vulnerabilities even with an unchanged memory layout are missed, leading to incomplete testing and low test reliability.

[0016] Therefore, this invention provides a testing method for an application, which will be described in detail below.

[0017] Figure 1 This is a flowchart of the steps of a testing method for an application provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the testing method for this application may include the following steps: Step 101: In a scenario where the memory address is fixed, obtain the variant test cases of the program under test, and test the program under test based on the variant test cases to obtain the first test result.

[0018] Step 102: Determine the variant test cases that represent abnormal outputs in the first test result as candidate test cases.

[0019] Step 103: In the scenario of randomized memory address layout, test the program under test based on the candidate test cases and obtain the second test result.

[0020] Step 104: Based on the second test result, filter from the candidate test cases and output the target test cases; the target test cases are used to characterize the security vulnerabilities existing in the program under test.

[0021] In this embodiment of the invention, the program under test can be any application that needs to be tested. The testing of the program under test can be divided into two stages. Steps 101-102 can be considered as the first stage, and steps 103-104 can be considered as the second stage. The first stage is conducted in a scenario where memory addresses are fixed (i.e., the address layout in memory is fixed and does not change), and the second stage is conducted in a scenario where memory address layout is randomized (i.e., the address space layout in memory changes randomly). The scenario where memory addresses are fixed corresponds to the case where Address Space Layout Randomization (ASLR) is not enabled, and the scenario where memory address layout is randomized corresponds to the case where ASLR is enabled. ASLR is a function provided by the operating system security mechanism. When ASLR is enabled, when the application starts, the operating system randomly allocates base addresses for the application's code segment, heap, stack, and other memory regions, making the memory layout of the application different each time it starts, thus increasing the difficulty for attackers to exploit memory vulnerabilities. However, this makes it difficult to reproduce the same vulnerability stably in the scenario where memory address layout is randomized, leading to a lower abnormal trigger rate of the program under test during testing, and thus reducing testing efficiency.

[0022] Therefore, in this embodiment of the invention, a first stage is performed. Specifically, in a scenario with a fixed memory address, the program under test is first tested using mutation test cases to obtain the first test results corresponding to the mutation test cases. Each mutation test case corresponds to a first test result. If the first test result indicates normal output, it means that the program under test will not exhibit abnormal behavior when the mutation test case is used as input, and the program under test can run stably. If the first test result indicates abnormal output, it means that the program under test will exhibit abnormal behavior when the mutation test case is used as input, causing the program under test to malfunction and run unstablely. Therefore, mutation test cases with abnormal output as their first test results can be identified as candidate test cases. Since the address space layout is fixed in the first stage, vulnerabilities that require a fixed memory layout to be triggered can be avoided, allowing candidate test cases to include mutation test cases that trigger vulnerabilities when the memory layout is fixed.

[0023] Furthermore, since ASLR is enabled by default in real-world usage scenarios, candidate test cases may include variant test cases that would not trigger vulnerabilities in actual usage scenarios. Therefore, in this embodiment of the invention, a second stage can be performed. Specifically, in a scenario with randomized memory address layout, the program under test can be tested again based on the candidate test cases to obtain a second test result. If the second test result indicates normal output, it means that in a scenario with changed address space layout, using the candidate test case as input will not trigger abnormal behavior in the program under test, and the program under test can run stably. If the second test result indicates abnormal output, it means that in a scenario with changed address space layout, using the candidate test case as input will also trigger abnormal behavior in the program under test, causing the program under test to malfunction. Therefore, in this embodiment of the invention, the candidate test cases can be verified based on the second test result to filter out test cases that would also trigger vulnerabilities in actual usage scenarios, and these test cases can be used as target test cases and output.

[0024] For example, assuming a given test set contains 100 variant test cases, in a scenario with a fixed address space layout, these variant test cases are used to test the program under test, obtaining the first test result for each of the 100 variant test cases. Further, if 60 of these variant test cases are found to have abnormal outputs in their first test results, these 60 variant test cases are selected as candidate test cases for the second phase of testing. In the second phase, in a scenario with a changing address space layout, these 60 candidate test cases are used to test the program under test again, obtaining the second test result for each of the 60 candidate test cases. Based on the second test results of these 60 candidate test cases, target test cases that might be missed in real-world usage scenarios are identified.

[0025] In summary, the application testing method provided by this invention, in scenarios with fixed memory addresses, obtains variant test cases for the program under test, and tests the program under test based on these variant test cases to obtain a first test result. Variant test cases whose abnormal output is represented by the first test result are identified as candidate test cases. In scenarios with randomized memory address layouts, the program under test is tested based on these candidate test cases to obtain a second test result. Based on the second test result, target test cases are selected from the candidate test cases and output; these target test cases are used to characterize security vulnerabilities in the program under test. Thus, the application under test is first tested in scenarios with fixed memory addresses, and variant test cases that trigger vulnerabilities when the memory layout remains unchanged are selected as candidate test cases. Further testing is then conducted in scenarios with randomly changing memory address layouts, and test cases that also trigger vulnerabilities in actual use scenarios are selected from the candidate test cases and output as target test cases. This avoids omitting variant test cases that trigger vulnerabilities in the output test cases, ensuring comprehensive testing and improving testing reliability.

[0026] Furthermore, by avoiding including test cases in the output that would not trigger vulnerabilities in actual use scenarios, the accuracy of the output results is ensured.

[0027] Optionally, in embodiments of the present invention, before the step of obtaining the variant test cases of the program under test in the scenario of fixed memory address described above, the following steps may be performed: Step S21: Modify the value of the randomization parameter to the first value using the first parameter modification command to enable the scenario with the fixed memory address.

[0028] In this embodiment of the invention, before testing the program under test based on the candidate test cases in the above-described scenario of randomized memory address layout, the following steps may also be performed: Step S31: Modify the value of the randomization parameter to a second value using the second parameter modification command to indicate the activation of the memory address layout randomization scenario.

[0029] The application testing method provided in this embodiment of the invention can be performed in a testing environment provided by the electronic device. Specifically, the testing method of the application can be executed by a testing engine tool in the testing environment. The testing engine tool can be selected as needed; for example, it can be AFL, libFuzzer, etc., and this embodiment of the invention does not limit this selection. The testing environment is a specially configured operating system environment for testing, thus avoiding any impact on the real deployment environment during the testing process.

[0030] Randomization parameters are provided by the operating system to control whether ASLR (Autonomous System Replication) is enabled or disabled. Both the first and second parameter modification commands are used to modify the randomization parameters, thereby controlling whether ASLR is enabled or disabled. The first parameter modification command carries a first value, and the second parameter modification command carries a second value. The parameter modification command can be a system command provided by the operating system, the first value can be a value defined by the operating system indicating that ASLR is disabled, and the second value can be a value defined by the operating system indicating that ASLR is enabled.

[0031] For example, the command format for modifying parameters can be: echo X | sudo tee Y Here, X represents the value carried by the parameter modification command, and Y represents the path of the randomization parameter carried in the parameter modification command. The randomization parameter can be an environment configuration file provided by the operating system, which can be considered as an ASLR on / off switch. The path of the randomization parameter is the path of this environment configuration file. By writing a first value to this environment configuration file, the parameter value of the randomization parameter is modified to the first value, thus disabling ASLR. By writing a second value to this system file, the parameter value of the randomization parameter is modified to the second value, thus enabling ASLR.

[0032] For example, taking a Linux system as an example, the path to the randomization parameter can be: / proc / sys / kernel / randomize_va_space. The first value can be 0, and the second value can be 2. Accordingly, the first reference modification command can be: echo 0 | sudo tee / proc / sys / kernel / randomize_va_space The second reference modification command can be: echo 2 | sudo tee / proc / sys / kernel / randomize_va_space In the first stage, the first value and the path of the randomization parameter can be written into the command format of the parameter modification command to obtain the first parameter modification command. Then, the first parameter modification command is executed to modify the value of the randomization parameter to the first value, thereby disabling the ASLR function of the operating system and enabling a scenario with fixed memory addresses, ensuring that the memory layout of the target program is consistent every time it runs.

[0033] In the second stage, the second value and the path of the randomization parameter can be written into the command format of the parameter modification command to obtain the second parameter modification command. Then, the second parameter modification command is executed to modify the value of the randomization parameter to the second value, thereby enabling the ASLR function of the operating system to enable the scenario of randomized memory address layout. This allows the target program to have a randomized memory layout each time it runs, thus simulating a real deployment environment.

[0034] In this embodiment of the invention, without relying on instrumentation of the program under test or modification of the testing engine tools, the address space layout change function can be turned on or off by modifying the value of the randomization parameter to a first value or a second value using a first parameter modification command and a second parameter modification command. This allows the program under test to be placed in either a scenario with fixed memory addresses or a scenario with randomized memory address layout, thereby switching between the entered scenarios. Therefore, the switching efficiency is higher.

[0035] Optionally, in this embodiment of the invention, the step of testing the program under test based on the mutation test cases to obtain a first test result may specifically include: Step 1011: Perform multiple rounds of mutation operations based on the preset mutation strategy and the seed test set of the program under test to generate multiple mutation test cases.

[0036] Step 1012: For any of the aforementioned variant test cases, use the variant test cases as input to the program under test, and run the program under test.

[0037] Step 1013: Determine the first test result based on the running results output by the program under test.

[0038] In this embodiment of the invention, the preset mutation strategy can be a strategy to adjust the expected input of the program under test to obtain unexpected input. The expected input is input data that conforms to the expected format and range of the program under test. Initially, the seed test set of the program under test includes at least one test case representing the expected input. The preset mutation strategy can be pre-set. For example, suppose the program under test is used to parse an input JPEG format image into image data in memory (e.g., a pixel array). Then, the expected input of the program under test can be an image file conforming to the JPEG format. The preset mutation strategy can include a numerical adjustment strategy and a byte modification strategy. The numerical adjustment strategy is used to modify the numerical values ​​in the JPEG format image file to values ​​exceeding the normal range, and the byte modification strategy is used to insert illegal characters into the JPEG format image file.

[0039] In one round of mutation operations, each test case in the seed test set can be mutated using various preset mutation strategies to obtain at least one unexpected input as a mutated test case. Correspondingly, the program under test can be fuzz-tested using at least one unexpected input generated in this round of mutation operations to obtain a first test result. The number of unexpected inputs generated in one round of mutation operations can be m×n, where m is the number of preset mutation strategies and n is the number of test cases currently included in the seed test set.

[0040] Suppose that p mutation test cases are generated in this round of mutation operations, then these p mutation test cases are used to perform a round of testing on the program under test. Specifically, for any one of these p mutation test cases, the mutation test case is used as input to the program under test, and the program under test is run. For example, the program under test can be controlled to take the mutation test case as input and run. For example, the actual output type of the program under test can be compared with the expected output type. If they match, the running result is considered abnormal; if they do not match, the running result is considered normal. For example, assuming the expected output type is an array of pixel values, but the actual output type is detected as a string, then the running result can be considered abnormal, and the corresponding signal type identifier can be adjusted. If the running result is abnormal, for example, the running result is a crash signal output by the program under test, then a test result representing the abnormal output is generated as the first test result corresponding to the mutation test case. Otherwise, a test result representing the normal output is generated as the first test result corresponding to the mutation test case. The specific forms of the test results representing abnormal output and the test results representing normal output can be preset. For example, the test result characterizing abnormal output may include the signal type identifier of the crash signal, and the test result characterizing normal output may be the default identifier, for example, 1. This embodiment of the invention does not limit this. The default identifier and the signal type identifier do not overlap. For example, the signal type identifier of the crash signal can be used as the test result characterizing abnormal output, and the default identifier (e.g., 1) can be used as the test result characterizing normal output.

[0041] In this embodiment, test cases are associated and stored with their corresponding test results. For example, this can be done in a table format, where each row includes two elements: the test case identifier and the corresponding test result. Alternatively, the test case identifier can be used as the key, and the corresponding test result as the key-value pair, stored as key-value pairs. For instance, if the test case identifier is "case 001" and the corresponding test result is "SIGSEGV", then key-value pairs can be used.<case 001,SIGSEGV> Stored in the manner described above.

[0042] Accordingly, if the test engine tool detects that the first test result includes a signal type identifier indicating a crash signal, it determines that the first test result represents abnormal output; if it detects that the first test result is a default identifier, it determines that the first test result represents normal output. The output type is compared with the expected output type; if they match, it indicates normal output.

[0043] In scenarios where address space layout changes, the memory layout of the program under test changes each time it runs. It may take a very long time for memory layout overlap to occur, making it extremely difficult to trigger potential vulnerabilities that only occur with a fixed memory layout. This results in low efficiency in discovering variant test cases that trigger vulnerabilities and a low crash trigger rate. In this embodiment of the invention, in scenarios with a fixed address space layout, the memory layout of the program under test remains unchanged each time it runs. With the same program input (i.e., for a single program input), the program's output is fixed each time it runs. Therefore, by using variant test cases as input to the program under test, only one run is needed to identify whether the variant test case will trigger a vulnerability under a fixed memory layout. This results in a higher crash trigger rate and allows for rapid, stable, and efficient triggering of potential vulnerabilities that only occur with a fixed memory layout. It also efficiently discovers variant test cases that trigger vulnerabilities under a fixed memory layout, thereby improving testing efficiency.

[0044] Of course, to improve reliability, variant test cases can be input into the program under test multiple times and run multiple times. If at least one of the runs results in an abnormal result, a first test result representing the abnormal output is generated. If multiple runs do not result in an abnormal output, a first test result representing the normal output is generated. This embodiment of the invention does not impose any restrictions on this approach.

[0045] Optionally, the steps of performing multiple rounds of mutation operations based on a preset mutation strategy and the seed test set of the program under test may specifically include: Step 1011a: For any round of mutation operation, modify the test cases in the seed test set using the preset mutation strategy to generate the mutation test cases corresponding to one round of mutation operation.

[0046] Step 1011b: Obtain the target coverage rate corresponding to each of the mutated test cases, and add the mutated test cases to the seed test set if the target coverage rate meets the preset conditions.

[0047] The target coverage rate is the code coverage rate during the execution of the program under test when the variant test cases are used as inputs.

[0048] In this embodiment of the invention, the seed test set only includes test cases corresponding to the expected input in the first round of mutation operations. Specifically, for any preset mutation strategy, each test case currently included in the seed test set can be modified according to the modification method represented by the preset mutation strategy. For example, assuming that the seed test set initially includes two test cases and the number of preset mutation strategies is 2, then 4 mutated test cases will be generated in the first round of mutation operations.

[0049] Specifically, corresponding to the aforementioned example, the numerical values ​​of the JPEG image files represented by these two test cases can be modified according to a numerical adjustment strategy to obtain two variant test cases. For example, assuming the JPEG image files represented by these two test cases are image file 1 and image file 2, for either image file 1 or image file 2, a numerical modification function can be called to modify the field values ​​of the image file's attribute fields to values ​​outside the normal range. For example, attribute fields may include image width and image height. Assuming the normal range for image width and image height is 100~200, the numerical modification function randomly selects a value outside the normal range as the field value for image width, and randomly selects a value outside the normal range as the field value for image height, thus obtaining a variant image file. For example, the field value of the image width field can be set to 65530. Furthermore, according to a byte modification strategy, illegal characters can be randomly inserted into the JPEG image files represented by these two test cases to obtain two variant test cases. For either image file 1 or image file 2, a character insertion function can be called to randomly select illegal characters from a preset illegal character set and insert them into the data portion of that image file, thus obtaining a mutated image file. The preset illegal character set consists of special characters predefined by the developers that are not supported by the program under test; for example, the character 0xFF representing an escape sequence, the character 0xD9 representing the end-of-image marker, etc. Compared to the original image file, the mutated image file contains illegal characters or abnormal field values. After generating the mutated image file, the hash value of the mutated image file and the hash value of the original image file can be calculated separately. If the two hash values ​​are inconsistent, the mutated image file is confirmed.

[0050] Furthermore, in this embodiment of the invention, the data format used by the variant test cases is the same as the expected input data format of the program under test. For example, the data format used by the variant test cases can be an image format. The variant test cases can be stored in a first specified directory, which can be pre-set by an engineer. The test engine tool can execute a calling command, inputting the path of the first specified directory into the program under test, and running the program under test. For example, the calling command can be `execv(path to the first specified directory, name of the program under test)`. For any of the four variant test cases, use that variant test case as input to the program under test (DUT) and run the DUT. Accordingly, the target coverage of that variant test case can be determined based on the code coverage during the execution. Code coverage is an indicator that measures the degree to which a test case covers the program's code. For example, the code coverage during a single run of the DUT can be the ratio of the number of branches executed during this run to the total number of branches included in the DUT. In this embodiment, this is specifically calculated using a test engine tool.

[0051] Furthermore, if the target coverage meets preset conditions, the mutated test case can be added to the seed test set. These preset conditions can be pre-set by testers based on experience. For example, preset conditions may include a target coverage greater than a preset coverage threshold, or an increase in target coverage (e.g., the target coverage corresponding to this mutated test case is greater than the target coverage determined for other mutated test cases in the previous iteration), etc. This embodiment of the invention does not limit these conditions. For any round of mutation operations, the number of mutated test cases that can be generated in that round can be: the number of preset mutation strategies × the number of test cases in the seed test set. Assuming that one of these four mutated test cases is added to the seed test set, then the seed test set includes three test cases, and therefore, 2 × 3 = 6 mutated test cases can be generated in the second round of mutation operations.

[0052] In this embodiment of the invention, for any round of mutation operation, the test cases in the seed test set are modified using a preset mutation strategy to generate mutated test cases corresponding to one round of mutation operation. The target coverage rate corresponding to each mutated test case is obtained, and if the target coverage rate meets preset conditions, the mutated test cases are added to the seed test set. In this way, by repopulating the seed test set with mutated test cases whose target coverage rate meets preset conditions, the number of mutated test cases can be automatically expanded, thereby improving the comprehensiveness of the test to a certain extent.

[0053] Optionally, embodiments of the present invention may further include the following steps: Step S41: Insert a detection component into the source code file of the program under test, and compile the source code file after inserting the detection component to obtain the executable file corresponding to the program under test; the detection component is used to detect the code coverage during the operation of the program under test.

[0054] Accordingly, the steps for obtaining the target coverage corresponding to each of the aforementioned variant test cases may specifically include: Step 1011b1: After the variant test case is input into the program under test and the program under test is executed, the code coverage detected by the detection component is taken as the target coverage.

[0055] In this embodiment of the invention, the source code files of the program under test and the code files of the detection component can be prepared in advance. The code files of the detection component are used to detect the code coverage during each run of the program under test. The source code files of the program under test and the code files of the detection component can be prepared in advance by technicians or downloaded from the network; this embodiment of the invention does not impose any restrictions on this. Next, the source code files of the program under test are compiled and instrumented by a test engine tool (e.g., AFL). Specifically, the code files of the detection component can be inserted into the source code files. For example, the afl-gcc or afl-clang-fast tools provided by the test engine tool can be used to insert the code files of the detection component into the source code files. Then, the compiler is invoked to compile the inserted source code files to obtain an executable program under test. Thus, during the run of the program under test, the detection component runs synchronously to detect the code coverage during this run.

[0056] For example, for any variant test case, when the variant test case is used as input to the program under test and the program under test after the variant test case input is run, the detection component can explore the program execution path and count the number of executed branches corresponding to the variant test case during each run. Finally, the ratio of the counted number of executed branches to the total number of branches in the code file is determined as the code coverage rate for this run. In this way, code coverage detection is efficiently achieved by inserting the detection component. Accordingly, the code coverage rate detected by the detection component can be obtained, and the code coverage rate for each run of the program under test can be obtained. Further, in one implementation, the maximum value of the code coverage rate in each run can be determined as the target coverage rate. Alternatively, in another implementation, the average value of the code coverage rate in each run can be calculated as the target coverage rate. This embodiment of the invention does not limit this.

[0057] In this embodiment of the invention, a detection component is inserted into the source code file of the program under test, and the source code file after the insertion of the detection component is compiled to obtain the executable file corresponding to the program under test. After the variant test cases are input into the program under test and the program under test is executed, the code coverage detected by the detection component is obtained, and the target coverage can be determined, thereby ensuring the efficiency of the target coverage determination to a certain extent.

[0058] It should be noted that the testing engine tool can also insert an exception handling component into the source code file. During each run, this component captures the execution results of the program under test. For example, the crash signal types can include SIGSEGV, SIGABRT, and SIGILL. A SIGSEGV type crash signal can be captured when the program under test outputs memory errors, such as illegal memory access or stack overflow. A SIGABRT type crash signal can be captured when an assertion failure occurs. A SIGILL type crash signal can be captured when an illegal instruction occurs.

[0059] Optionally, in this embodiment of the invention, the target test case includes a first target test case and a second target test case; the step of filtering from the candidate test cases based on the second test result may specifically include: Step 1041: Determine the candidate test cases that represent abnormal outputs of the second test result as test cases to be verified.

[0060] Step 1042: If the second test result of the test case to be verified is consistent with the exception type represented by the first test result of the test case to be verified, then the test case to be verified is determined as the first target test case.

[0061] Step 1043: Otherwise, the test case to be verified is determined as the second target test case.

[0062] In this embodiment of the invention, in the first stage, the mutated test cases that represent abnormal outputs of the first test results and their first test results (including crash type identifiers) can be recorded, and all the mutated test cases that represent abnormal outputs of the first test results constitute a candidate test case set S1.

[0063] In the second phase, vulnerability exploitability verification can be performed on each candidate test case in the candidate test case set S1 to determine whether the vulnerability triggered by the candidate test case can still be triggered in a real-world scenario where ASLR is enabled. Specifically, the program under test can be tested based on the candidate test cases to obtain the second test results.

[0064] Specifically, this can include: for any candidate test case, using the candidate test case as input to the program under test and running the program under test; if the running result is an abnormal result, for example, a crash signal output by the program under test, then generating a test result representing the abnormal output as the second test result corresponding to the candidate test case. Otherwise, generating a test result representing normal output as the second test result corresponding to the candidate test case. Accordingly, if the test engine tool detects that the second test result includes a signal type identifier of a crash signal, it determines that the second test result represents abnormal output; if it detects that the second test result has a default identifier, it determines that the second test result represents normal output.

[0065] Furthermore, if the second test result corresponding to the candidate test case indicates normal output, it can be considered that the vulnerability triggered by the candidate test case will not be triggered in a real scenario where ASLR is enabled. The exploitability of the vulnerability in the candidate test case is low, and its subsequent verification priority is low. If the second test result corresponding to the candidate test case indicates abnormal output, it indicates that its subsequent verification priority is high. Therefore, the candidate test case is identified as a test case to be verified and further verification is carried out.

[0066] Specifically, for any test case to be verified, it can be checked whether the signal type identifier included in the second test result of the test case is the same as the signal type identifier included in the first test result of the test case. The anomaly type represented by the test result refers to the type of crash signal indicated by the signal type identifier included in the test result. One signal type identifier represents one type of crash signal; for example, the signal type identifier may include "SIGSEGV", "SIGABRT", or "SIGILL". If the two signal type identifiers are the same, it is determined that the anomaly types represented by the two are consistent. Conversely, if the two signal type identifiers are different, it is determined that the anomaly types represented by the two are inconsistent.

[0067] Furthermore, if the anomaly types represented by the two are consistent, it can be assumed that the vulnerability triggered by the candidate test case will be triggered in a real-world scenario where ASLR is enabled. This indicates that the candidate test case is not affected by changes in memory address layout, and the corresponding vulnerability has higher exploitability. Accordingly, the test case to be verified can be identified as the first target test case. For example, assuming that the signal type identifiers included in the first test result and the second test result of the test case to be verified, a, are both "SIGSEGV", then the test case to be verified, a, can be identified as the first target test case. If the anomaly types represented by the two are inconsistent, it can be assumed that the vulnerability triggered by the candidate test case will be triggered in a real-world scenario where ASLR is enabled. This indicates that the candidate test case is affected by changes in memory address layout and is more sensitive to changes in memory address layout than the first target test case in a scenario with randomized memory address layout. Therefore, the vulnerability exploitability of the candidate test case is lower than that of the first target test case. Accordingly, the test case to be verified can be identified as the second target test case.

[0068] In this embodiment of the invention, the first target test case triggers abnormal operation of the program under test (SUPT) with ASLR functionality disabled or enabled, and utilizes more advanced test cases. The second target test case triggers abnormal operation of the SUPT with ASLR functionality disabled or enabled, and utilizes less advanced test cases. The SUPT is more likely to be triggered into abnormal behavior by the first target test case. The first and second target test cases can be stored in a second designated directory, which can be pre-set by technical personnel. The second designated directory is different for different SUPTs. The test engine tool can determine the corresponding second designated directory for each SUPT by mapping the SUPT name to the second designated directory, and then obtain the first and second target test cases associated with that SUPT from the corresponding second designated directory.

[0069] In this embodiment of the invention, in scenarios where the address space layout changes, the program under test is tested based on candidate test cases to obtain a second test result. Based on the second test result, the exploitability of the candidate test cases mined in the first stage in real-world scenarios is actively verified. The final target test cases are further filtered from the candidate test cases. This avoids outputting a large number of theoretical vulnerabilities that do not pose a security risk in real-world scenarios and cannot be exploited, thus reducing unnecessary analysis work for security personnel and lowering their processing costs. While maintaining testing efficiency, this method ensures that the final selected target test cases are those that will trigger vulnerabilities in real-world scenarios, effectively verifying the actual exploitability of the vulnerabilities and ensuring the reference value of the final target test cases output to security personnel, thereby improving the practicality of the testing method.

[0070] Optionally, the step of outputting the target test cases described above in this embodiment of the invention may specifically include: Step 1044: Set a first processing identifier for the first target test case and set a second processing identifier for the second target test case; the processing priority represented by the first processing identifier is higher than the processing priority represented by the second processing identifier; Step 1045: Output the first target test case and the second target test case, which carry the first processing identifier and the second processing identifier, respectively.

[0071] In this embodiment of the invention, the high-risk level identifier includes a first processing identifier and a second processing identifier. The first processing identifier can be predefined as "high-1", and the second processing identifier can be predefined as "high-2". The first and second processing identifiers are used to characterize test cases as highly exploitable vulnerabilities, indicating to security personnel that the vulnerabilities caused by these test cases have a higher priority for response and require priority analysis and optimization based on these test cases. For example, if the number of target test cases exceeds a preset threshold, it indicates that the program under test has poor robustness when facing abnormal input, is prone to crashing, and has low security. Security personnel can then perform targeted optimization based on the target test cases to improve security.

[0072] For example, the first target test case and the second target test case corresponding to the program under test can be obtained from the second designated directory corresponding to the program under test. A first processing identifier is marked at the beginning of the first target test case. A second processing identifier is marked at the beginning of the first and second target test cases. The processing priority represented by the first processing identifier is higher than the processing priority represented by the second processing identifier, so that security personnel can prioritize processing the first target test case, which has the same anomaly type, higher vulnerability exploitability, and higher risk, in order to eliminate the risk as soon as possible.

[0073] Next, all target test cases are added to the same set as the verification result and output to security personnel. For example, in one implementation, this set can be displayed on a screen for output. Alternatively, in another implementation, the set can be sent to an electronic terminal used by security personnel, or, in yet another, the set can be output to a user-specified output directory. This embodiment of the invention does not limit the scope of the implementation.

[0074] In this embodiment of the invention, by setting a high-risk level identifier for the target test cases and outputting the target test cases carrying the high-risk identifier, security personnel can quickly learn that these test cases will trigger exploitable vulnerabilities based on the high-risk level identifier, and thus quickly evaluate and optimize the program under test.

[0075] Optionally, embodiments of the present invention may further include the following steps: Step S51: Determine the candidate test cases that represent normal output based on the second test result as low-risk test cases.

[0076] Step S52: Set a low-risk flag for the low-risk use case and then output the result.

[0077] In this embodiment of the invention, the low-risk level identifier can be predefined as "low". The low-risk level identifier is used to characterize test cases as low-exploitability vulnerabilities, indicating to security personnel that the vulnerabilities caused by these test cases have a lower response priority, for their reference. Specifically, for test cases that do not cause program crashes in the second phase (i.e., candidate test cases whose second test results indicate normal output), it can be considered that the vulnerabilities caused by these test cases are sensitive to the ASLR function, and their exploitability is low when the ASLR function is enabled. However, since these test cases cause the tested program to malfunction, these test cases can be further classified as low-risk test cases and output.

[0078] The second test result, representing candidate test cases with normal output, can be stored in a third designated directory. This third designated directory can be pre-set by technical personnel, and different third designated directories exist for different programs under test. The test engine tool can determine the corresponding third designated directory for each program under test by mapping its name to the third designated directory. Then, it can obtain the candidate test cases representing normal output associated with that program under test from that corresponding third designated directory, using them as low-risk test cases. Specifically, a low-risk level identifier can be marked at the beginning of each low-risk test case. Then, all low-risk test cases carrying the low-risk identifier are added to the same set and output to security personnel. For example, in one implementation, this set can be displayed on a screen for output. Alternatively, in another implementation, the set can be sent to an electronic terminal used by security personnel. Or, in yet another implementation, the set can be output to a user-specified output directory. This embodiment of the invention does not limit the scope of these implementations.

[0079] In this embodiment of the invention, by setting a low-risk level identifier for low-risk test cases and outputting low-risk test cases carrying the low-risk identifier, security personnel can quickly determine that these test cases are only used for auxiliary analysis based on the low-risk level identifier. This provides security personnel with more reference information while enabling them to quickly distinguish the risk level of the output test cases, improving the accuracy of vulnerability response prioritization.

[0080] It should be noted that, in one implementation, in scenarios where the address space layout changes, the variant test cases that represent normal output in the first test result can be further tested, and the variant test cases that represent abnormal output in the test result can be further output to security personnel to provide more reference information and further ensure the comprehensiveness of the test.

[0081] Figure 2 This is a schematic diagram of a processing flow provided by an embodiment of the present invention, such as... Figure 2 As shown, in the first stage, the source code file of the program under test can be obtained first. Then, ASLR is disabled using the first parameter modification command, fixing the memory address corresponding to the runtime of the program under test. Next, the source code file of the program under test is compiled and instrumented using a test engine tool to obtain the program under test. Then, the test engine tool is run, which obtains variant test cases for the program under test and tests the program under test based on these variant test cases to obtain the first test result. For example, the test engine tool can be run using a preset startup command. For example, the preset startup command can be `afl-fuzz -i input -ooutput`. Here, the `input` parameter can be used to specify the seed test set in the initial state, and the `output` parameter can be used to specify the output directory. Further, candidate test cases can be determined based on the variant test cases corresponding to the first test results with crash signals, thus obtaining a candidate test case set. Specifically, variant test cases that represent abnormal output in the first test result can be identified as candidate test cases, and the candidate test cases and their first test results (including crash type identifiers) are recorded. All candidate test cases constitute the candidate test case set.

[0082] Next, the candidate test case set and its first test result (including crash type identifier) ​​are passed to the second stage for verification. In the second stage, ASLR is first enabled by modifying the second parameter, causing the memory address layout corresponding to the program under test to change randomly during runtime. Then, a candidate test case is taken from the candidate test case set, and the program under test is tested based on this candidate test case to obtain a second test result for further screening of candidate test cases. If the second test result indicates abnormal output, the corresponding candidate test case is determined as the target test case. If the second test result indicates normal output, the corresponding candidate test case is determined as a low-risk test case. Next, if there are still unverified candidate test cases in the candidate test case set, the process returns to the step of taking another candidate test case from the candidate test case set to continue execution. If there are no unverified candidate test cases in the candidate test case set, the test cases are output. Specifically, the target test case with a high-risk identifier and the low-risk test case with a low-risk identifier can be output.

[0083] In this embodiment of the invention, a two-stage execution mode is adopted for fuzz testing. The fuzz testing process is divided into two stages: efficient vulnerability discovery and realistic verification. In the first stage with ASLR disabled, potential vulnerabilities that can only be triggered with a fixed memory layout are efficiently triggered. This efficiently discovers variant test cases that trigger vulnerabilities when the memory layout remains unchanged, thus improving testing efficiency. Simultaneously, in the second stage with ASLR enabled, the exploitability of the candidate test cases discovered in the first stage is verified. In this way, while ensuring testing efficiency, the problem of outputting a large number of theoretical vulnerabilities that do not pose a security risk in real-world scenarios and cannot be exploited is avoided, which would increase the unnecessary analysis work for security personnel, thereby reducing the processing cost for security personnel.

[0084] Reference Figure 3 The diagram illustrates a block diagram of an application testing apparatus provided in an embodiment of the present invention, such as... Figure 3 As shown, the testing equipment for this application may specifically include: The first test module 201 is used to obtain the variant test cases of the program under test in a scenario with fixed memory address, and to test the program under test based on the variant test cases to obtain the first test result. The first determining module 202 is used to determine the variant test cases whose first test results represent abnormal output as candidate test cases; The second test module 203 is used to test the program under test based on the candidate test cases in a scenario of randomized memory address layout, and obtain the second test result. The first output module 204 is used to filter from the candidate test cases based on the second test result and output the target test cases; the target test cases are used to characterize the security vulnerabilities existing in the program under test.

[0085] Optionally, the device further includes: a first modification module, used to modify the parameter value of the randomization parameter to a first value through a first parameter modification command, so as to enable the scenario with the fixed memory address; The device further includes a second modification module, used to modify the parameter value of the randomization parameter to a second value through a second parameter modification command, so as to indicate the scenario of enabling the randomization of the memory address layout.

[0086] Optionally, the target test cases include a first target test case and a second target test case; The first output module 204 is specifically used for: The candidate test cases whose output is abnormal due to the second test result are identified as test cases to be verified. If the second test result of the test case to be verified is consistent with the exception type represented by the first test result of the test case to be verified, then the test case to be verified is determined as the first target test case; Otherwise, the test case to be verified is determined as the second target test case.

[0087] Optionally, the first test module 201 is specifically used for: Based on the preset mutation strategy and the seed test set of the program under test, multiple rounds of mutation operations are performed to generate multiple mutation test cases respectively. For any of the aforementioned variant test cases, the variant test cases are used as inputs to the program under test, and the program under test is run. The first test result is determined based on the running results output by the program under test.

[0088] Optionally, the first test module 201 is further configured to: For any round of mutation operation, the test cases in the seed test set are modified using the preset mutation strategy to generate the mutated test cases corresponding to one round of mutation operation; Obtain the target coverage corresponding to each of the mutated test cases, and add the mutated test cases to the seed test set if the target coverage meets the preset conditions.

[0089] Optionally, the device further includes: The processing module is used to insert a detection component into the source code file of the program under test, and to compile the source code file after inserting the detection component to obtain the executable file corresponding to the program under test; the detection component is used to detect the code coverage during the execution of the program under test. The first test module 201 is further configured to: after the variant test cases are input to the program under test and the program under test is executed, use the code coverage detected by the detection component as the target coverage.

[0090] Optionally, the first output module 204 is further configured to: set a first processing identifier for the first target test case and set a second processing identifier for the second target test case; the priority represented by the first processing identifier is higher than the priority represented by the second processing identifier; Output the first target test case and the second target test case, each carrying the first processing identifier and the second processing identifier, respectively; The device further includes: a second determining module, configured to determine candidate test cases whose output is normal as represented by the second test result as low-risk test cases; and a second output module, configured to set a low-risk flag for the low-risk test cases and then output them.

[0091] In summary, the application testing device provided in this embodiment of the invention obtains variant test cases for the program under test in scenarios with fixed memory addresses, and tests the program under test based on the variant test cases to obtain a first test result. Variant test cases whose abnormal output is represented by the first test result are determined as candidate test cases. In scenarios with randomized memory address layouts, the program under test is tested based on the candidate test cases to obtain a second test result. Based on the second test result, target test cases are selected from the candidate test cases and output; the target test cases are used to characterize the security vulnerabilities existing in the program under test. Thus, the application under test is first tested in scenarios with fixed memory addresses, and variant test cases that trigger vulnerabilities when the memory layout remains unchanged are used as candidate test cases; further, testing is performed again in scenarios with randomly changing memory address layouts, and test cases that also trigger vulnerabilities in actual use scenarios are selected from the candidate test cases and output as target test cases. This avoids omitting variant test cases that trigger vulnerabilities in the output test cases, ensuring the comprehensiveness of the test and improving the reliability of the test.

[0092] Furthermore, by avoiding including test cases in the output that would not trigger vulnerabilities in actual use scenarios, the accuracy of the output results is ensured.

[0093] Reference Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. For example... Figure 4 As shown, the electronic device includes: a processor, a memory, a communication interface, and a communication bus.

[0094] The processor, the memory, and the communication interface communicate with each other via the communication bus; the memory stores executable instructions that cause the processor to execute the test method of the application program described in the foregoing embodiments. The executable instructions can form a program.

[0095] This invention provides a machine-readable medium storing instructions that, when executed by one or more processors, enable the processors to perform the testing method of the application described in the foregoing embodiments. The machine-readable medium may be one or more.

[0096] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0097] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0099] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing terminal device to operate in a predictive manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0103] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0104] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0105] The present invention has provided a detailed description of an application testing method, an application testing device, an electronic device, and a readable medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for testing an application, characterized in that, The method includes: In a scenario with fixed memory addresses, obtain the variant test cases of the program under test, and test the program under test based on the variant test cases to obtain the first test result; The variant test cases whose first test results represent abnormal outputs are identified as candidate test cases. In a scenario where memory address layout is randomized, the program under test is tested based on the candidate test cases to obtain a second test result; Based on the second test result, the candidate test cases are selected and the target test cases are output; the target test cases are used to characterize the security vulnerabilities existing in the program under test.

2. The method according to claim 1, characterized in that, The method further includes: modifying the value of the randomization parameter to a first value using a first parameter modification command to enable the scenario with the fixed memory address; The method further includes: modifying the value of the randomization parameter to a second value using a second parameter modification command to indicate the scenario of enabling the randomization of the memory address layout.

3. The method according to claim 1, characterized in that, The target test cases include a first target test case and a second target test case; Then, the step of filtering from the candidate test cases based on the second test result includes: The candidate test cases whose output is abnormal due to the second test result are identified as test cases to be verified. If the second test result of the test case to be verified is consistent with the exception type represented by the first test result of the test case to be verified, then the test case to be verified is determined as the first target test case; Otherwise, the test case to be verified is determined as the second target test case.

4. The method according to claim 1, characterized in that, The step of testing the program under test based on the variant test cases to obtain a first test result includes: Based on the preset mutation strategy and the seed test set of the program under test, multiple rounds of mutation operations are performed to generate multiple mutation test cases respectively. For any of the aforementioned variant test cases, the variant test cases are used as inputs to the program under test, and the program under test is run. The first test result is determined based on the running results output by the program under test.

5. The method according to claim 4, characterized in that, The process of performing multiple rounds of mutation operations based on a preset mutation strategy and the seed test set of the program under test includes: For any round of mutation operation, the test cases in the seed test set are modified using the preset mutation strategy to generate the mutated test cases corresponding to one round of mutation operation; Obtain the target coverage corresponding to each of the mutated test cases, and add the mutated test cases to the seed test set if the target coverage meets the preset conditions.

6. The method according to claim 5, characterized in that, The method further includes: A detection component is inserted into the source code file of the program under test, and the source code file after the detection component is inserted is compiled to obtain the executable file corresponding to the program under test; the detection component is used to detect the code coverage during the execution of the program under test. Therefore, obtaining the target coverage corresponding to each of the mutated test cases includes: After the variant test cases are input into the program under test and the program under test is executed, the code coverage detected by the detection component is taken as the target coverage.

7. The method according to claim 3, characterized in that, The output target test cases include: A first processing identifier is set for the first target test case, and a second processing identifier is set for the second target test case; the priority represented by the first processing identifier is higher than the priority represented by the second processing identifier. Output the first target test case and the second target test case, each carrying the first processing identifier and the second processing identifier, respectively; The method further includes: identifying candidate test cases whose output is normal based on the second test result as low-risk test cases; and setting a low-risk flag for the low-risk test cases before outputting them.

8. A testing apparatus for an application, characterized in that, The device includes: The first testing module is used to obtain variant test cases of the program under test in a scenario with fixed memory address, and to test the program under test based on the variant test cases to obtain the first test result. The first determining module is used to determine the variant test cases whose abnormal output is characterized by the first test result as candidate test cases; The second testing module is used to test the program under test based on the candidate test cases in a scenario of randomized memory address layout, and obtain the second test result. The first output module is used to filter from the candidate test cases based on the second test result and output the target test cases; the target test cases are used to characterize the security vulnerabilities existing in the program under test.

9. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store executable instructions that cause the processor to perform the method as described in any one of claims 1 to 7.

10. A machine-readable medium, characterized in that, It stores instructions that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1 to 7.

Images