Test case generation method and device

Abstract

The invention discloses a test case generation method and device, and relates to the field of software testing, and the method comprises the steps: obtaining a first test case; the first test case is input into the to-be-tested software inserted with the probe to run, so that constraint information of the first test case is obtained, and the constraint information indicates the execution condition of the constraint passed by the first test case in the execution path of the code related to the target point; the target points comprise code lines with potential vulnerabilities in source codes of the to-be-tested software; and generating a second test case based on the first test case and constraint information of the first test case, wherein the constraint information is used for indicating a variation mode of the first test case. Through the method, the first test case can be varied according to the constraint on the execution path related to the target point, and exploration on irrelevant paths which cannot reach the target point is reduced, so that the variation efficiency of the test case capable of triggering the target point is improved.

Description

Technical Field

[0001] This application relates to the field of software testing, and in particular to a test case generation method and apparatus. Background Technology

[0002] With the development of automated software testing, more powerful detection tools and verification methods are needed to ensure the safety and quality of automated software testing technologies, particularly for problematic code snippets. Fuzz testing is a popular dynamic testing technique that automatically generates a massive number of randomized test cases, inputs them into the dynamically running software under test, triggers software exceptions, and discovers software defects (bugs). Therefore, by providing actual inputs to trigger exceptions, fuzz testing can guarantee an extremely low false positive rate. Depending on the level of understanding of the testing objective, fuzz testing can be divided into black-box, white-box, and gray-box fuzz testing. Among these, gray-box fuzz testing is the most scalable and practical software testing method. Because there is a strong correlation between code coverage and bugs, most gray-box fuzz testing tools are coverage-guided.

[0003] However, since most of the covered code may not contain bugs, blindly expanding code coverage is inefficient. This leads to excessive time spent exploring irrelevant paths in the test cases generated by existing solutions, slowing down the generation of test cases that trigger crashes at specific target points. Summary of the Invention

[0004] This application provides a test case generation method and apparatus to reduce the exploration overhead of test cases on invalid paths and improve the mutation efficiency of test cases that trigger vulnerabilities in specific lines of code.

[0005] Firstly, this application provides a test case generation method, which includes: obtaining test cases (denoted as first test cases) of code to be tested (such as software to be tested), wherein the first test case may be an initial test case or a test case selected from a set of test cases; inserting the first test case into the software to be tested equipped with a probe and running it, thereby obtaining constraint information of the first test case collected by the probe, wherein the constraint information indicates the execution status of constraints traversed by the first test case in the critical path, wherein the critical path refers to the execution path of code related to the target point executed by running the first test case, and the target point includes lines of code in the source code of the software to be tested that may contain vulnerabilities; wherein the constraints include logical constraints in the source code (such as if or switch); and guiding the first test case to mutate based on the constraint information of the first test case to generate a second test case, wherein the constraint information of the first test case is used to indicate the mutation method of the first test case, such as instructing the first test case to mutate in the direction of resolving the constraints to be resolved in the first test case.

[0006] Through the above design, the first test case is guided to mutate based on the constraint information of the first test case. During the mutation process, the first test case can mutate on the constraints on the execution path related to the target point, reducing the exploration on irrelevant paths that cannot reach the target point, thereby improving the mutation efficiency of test cases that can trigger the target point.

[0007] In one possible design, the method further includes: determining functions in the source code related to the target point (functions related to the target point); inserting probes at the entry points of the functions related to the target point in the source code or in the functions, such as inserting probes before each branch entry point of the function. These probes are used to collect information on the execution of test cases for these constraints, such as whether the constraint is executed, which branch of the constraint is executed, and the attribute information of the constraint (such as constraint type, whether the constraint contains variables, constants, variable types, etc.); the source code with the probes inserted can be compiled to obtain an executable program of the software under test, and the step of inputting the first test case into the software under test after inserting the probes includes: inputting the first test case into the executable program for execution.

[0008] By inserting probe code into functions related to the target point in the source code, the test cases can collect information on the program execution path related to the target point, thereby reducing the collection and use of irrelevant information during the testing process.

[0009] In one possible design, determining the functions in the source code related to the target point includes: determining a data flow based on the source code and the target point, the data flow indicating variables related to the target point (relevant variables of the target point) and the relationships between the variables; determining data flow-related nodes based on the data flow; the data flow-related nodes include the lines of code in the source code where operations affecting the variables are located; the functions related to the target point include the functions containing the lines of code.

[0010] The above design allows for the identification of code relevant to the target point, preventing exploration on irrelevant paths during mutation.

[0011] In one possible design, the probe is further used to collect coverage information of the first test case, the coverage information indicating the lines of code covered by the first test case; the method further includes: obtaining information on historical constraints, the information on historical constraints indicating the priority of the historical constraints; determining new constraints in the first test case based on the constraint information of the first test case; determining the priority of the new constraints, the priority of the new constraints being determined based on the coverage information of the first test case and the score of the data flow-related nodes; the higher the correlation between the data flow-related nodes and the target point, the higher the score; the priority of the new constraints is higher than the priority of the historical constraints.

[0012] The above design evaluates the priority of constraints based on the correlation between data flow nodes and target points, avoiding the use of the distance between code lines and target points to evaluate constraint priority. This allows for more accurate location of test cases and constraints that are closer to triggering the target point, accelerating the generation of test cases that can trigger the target point and improving the efficiency of targeted test case generation.

[0013] In one possible design, the constraint information includes the association between the bytes included in the first test case and the constraints traversed in the key path; generating the second test case based on the first test case and its constraint information includes: determining a first mutation strategy based on the constraint information of the first test case and the association, the first mutation strategy indicating the mutation object and / or mutation method; the mutation object includes the bytes associated with the constraint to be resolved in the first test case, the mutation method indicating that the mutation object satisfies the condition of the constraint to be resolved; and mutating the first test case according to the first mutation strategy to generate the second test case.

[0014] The above design improves the efficiency of generating test cases in a targeted manner by guiding the mutation of constraint-related bytes in the test cases.

[0015] In one possible design, the probe is also used to collect the propagation path of the bytes included in the first test case in the lines of code covered by the first test case; the association is determined based on the propagation path.

[0016] In one possible design, the method further includes: obtaining descriptive information of the target point, the descriptive information including the vulnerability type of the target point and / or variables related to the target point; and generating a second test case based on the first test case and the descriptive information of the target point.

[0017] The above design guides test cases to mutate based on the description information of the target point, solving the problem that existing test engines cannot handle test cases that fail to trigger vulnerabilities even when they reach the target point.

[0018] In one possible design, the probe is further used to collect the propagation path of the bytes included in the first test case in the lines of code covered by the first test case; generating a second test case based on the description information of the first test case and the target point includes: determining a second mutation strategy based on the description information of the target point, the second mutation strategy indicating the mutation object and / or mutation method; the mutation object includes bytes associated with variables related to the target point, the mutation method indicating the operation of the mutation object that satisfies the vulnerability triggering condition of the target point; and mutating the first test case according to the first mutation strategy to generate the second test case.

[0019] In one possible design, obtaining the descriptive information of the target point includes: determining the vulnerability type of the target point and / or variables related to the target point based on the source code; or, obtaining static analysis data of the source code, wherein the static analysis data includes the vulnerability type of the target point and / or variables related to the target point.

[0020] In one possible design, the method further includes: inputting the second test case into the software under test into which the probe is inserted and running it to obtain constraint information of the second test case; and generating a third test case based on the second test case and the constraint information of the second test case.

[0021] In one possible design, the second test case has been executed to the target point; the method further includes: obtaining description information of the target point, the description information including the vulnerability type of the target point and / or variables related to the target point; generating a third test case based on the second test case and the description information of the target point; the description information of the target point is used to indicate the mutation mode of the second test case; the third test case is used to test the software under test.

[0022] Secondly, this application provides another test case generation method, which includes: obtaining test cases (denoted as second test cases) of the code to be tested (such as software to be tested); obtaining the vulnerability type and / or related variables of the target point in the source code to be tested; and guiding the second test case to mutate based on the vulnerability type and related variables of the target point to generate a third test case.

[0023] Through the above design, test cases are mutated based on the vulnerability type and / or related variables of the target point, which solves the problem that test cases that cannot be handled by existing test engines still fail to trigger vulnerabilities when they reach the target point.

[0024] Thirdly, this application also provides a test case generation device, which has the function of implementing the behavior in the method of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. And / or, the device has the function of implementing the behavior in the method of the second aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the device structure includes an acquisition module, a test module, and a generation module; optionally, it also includes a determination module. In one possible design, some or all of the modules can be the same module; for example, the acquisition module and the determination module can be the same module. These modules can perform the functions of the behavior in the method example of the first aspect described above, as detailed in the method example, and will not be repeated here.

[0025] Fourthly, this application also provides a computing device cluster, which includes at least one computing device having the functionality to implement the behavior in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. And / or, the at least one computing device has the functionality to implement the behavior in the method example of the second aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. Each computing device includes a processor and a memory. The processor is configured to support the computing device in executing the method described in the first aspect or any possible design of the first aspect. The memory is coupled to the processor and stores the necessary program instructions and data of the computing device. The computing device also includes a communication interface for communicating with other devices.

[0026] Fifthly, this application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect.

[0027] Sixthly, this application also provides a computer program product containing instructions that, when run on a computer, causes the computer to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect.

[0028] In a seventh aspect, this application also provides a computer chip connected to a memory, the chip being used to read and execute a software program stored in the memory, to execute the method described in the first aspect or any possible design of the first aspect, or to execute the method described in the second aspect or any possible design of the second aspect.

[0029] For the beneficial effects of aspects two through seven, please refer to the beneficial effects of aspect one, which will not be repeated here. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a variation process provided in an embodiment of this application;

[0031] Figure 2 This is a schematic diagram of a system architecture provided in an embodiment of this application;

[0032] Figure 3 This application provides a flowchart illustrating a selective pile driving method.

[0033] Figure 4 This is a flowchart illustrating a test case generation method provided in an embodiment of this application;

[0034] Figure 5 This is a flowchart illustrating another test case generation method provided in an embodiment of this application;

[0035] Figure 6A This is a flowchart illustrating another test case generation method provided in an embodiment of this application;

[0036] Figure 6B This is a flowchart illustrating another test case generation method provided in an embodiment of this application;

[0037] Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0038] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0039] Figure 9 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0040] Figure 10 This is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0041] To facilitate understanding of the technical solutions of this application, the terms involved in the embodiments of this application will be explained first.

[0042] 1. Software code, executable program

[0043] Software code is written by programmers using a language supported by development tools. Software code consists of a set of numbers or letters that have a meaning. Software code includes multiple lines of code, each with a line number to identify it.

[0044] Software code cannot be executed directly on a computer. It needs to be converted into binary code that the machine can recognize by calling the compiler based on the configuration information recorded in the build file. This binary code is also called an executable program.

[0045] 2. Functions / methods

[0046] A function / method is a subroutine within a class. A method typically consists of a series of statements and performs a specific function. In object-oriented programming, it's called a method; in procedural programming, it's called a function.

[0047] In programming, a function is a reusable block of code designed to perform a specific task. Think of it like a toolbox; when a task needs to be completed, you take out the corresponding tool (the function) and use it. Functions accept parameters, which are like the raw materials needed to use the tool. The function then performs the appropriate operations based on these parameters. Furthermore, functions typically return a result, which is like the finished product, which can be used for further processing or displayed to the user. By using functions, developers can create modular and reusable code. This helps reduce repetitive code writing, making the code more concise, maintainable, and easier to update.

[0048] When defining a function, you need to specify its name, parameter list, and function body. The function name is the identifier used to call the function, the parameter list specifies the input values ​​the function accepts, and the function body contains the actual code implementation. To call the function, simply use the function name followed by the necessary parameters. In general, functions are a very important concept in programming, making code more modular, reusable, and maintainable.

[0049] 3. Variables

[0050] Variables are used to store data values, and these values ​​can be changed during program execution. A variable consists of a variable name and a data type. The variable name identifies the variable, while the data type defines the data type that the variable can store. Data types define the types of data that a variable can store, such as integers, floating-point numbers, characters, strings, booleans, hash tables, pointers, etc.

[0051] Integer types: used to store integers, such as int, long, etc.

[0052] Floating-point types: used to store decimals, such as float, double, etc.

[0053] Character type: Used to store a single character, such as char.

[0054] String type: Used to store sequences of characters, such as String (in Java) or std::string (in C++).

[0055] Boolean type: used to store two states, true or false, such as bool.

[0056] 4. Constraints

[0057] Constraints can restrict the behavior of type parameters, variables, or functions. For example, logical constraints are used to limit logical conditions during program execution. In conditional statements (such as if and switch statements), logical constraints can be used to ensure that certain operations are performed only when specific conditions are met.

[0058] 5, if

[0059] The if statement is a very basic and important control flow statement used to determine whether to execute a block of code based on the truth value of a certain condition (Boolean expression). The basic syntax structure of the if statement is as follows:

[0060]

[0061] Here, the "condition" is a Boolean expression whose value is either true or false. If the condition is true, the code block within the curly braces {} is executed; if the condition is false, this code block is skipped, and execution continues with the code following the if statement.

[0062] Here is a simple example of an if statement used to determine whether a number is positive:

[0063]

[0064] In this example, if the value of number is greater than 0, then the condition number > 0 is true, and the program will output "This number is positive.". Otherwise, the program will not execute the printf statement. In this example, because the value of number is 5, the program will execute the printf statement, that is, output "This number is positive."

[0065] The if statement can also be combined with other control flow statements to form more complex logical structures, for example:

[0066] 1) if-else statement: used to execute another code block when the condition is false.

[0067]

[0068] 2) if-else if-else statement: used to check multiple conditions and execute the corresponding code block based on the first condition that is true.

[0069]

[0070]

[0071] 3) Nested if statements: One or more if statements are nested within an if or else if code block.

[0072]

[0073] Note that when using an if statement, you should ensure that the conditional expression is of Boolean type or can be implicitly converted to Boolean type. Curly braces {} are used to define the code block of the if statement.

[0074] 6. Switch

[0075] `switch` is a very useful control flow statement that allows a program to select one of several possible code blocks to execute based on the value of a variable. Imagine having many different conditions to evaluate, each corresponding to a different operation; a `switch` statement would be very convenient. It acts like a multiplexer, guiding the program's execution flow to different branches based on different input values. Generally, a `switch` statement consists of an expression (the value to be evaluated) and one or more `case` clauses (each containing a value and a code block to be executed). When the value of the expression matches the value in a `case` clause, the code in that `case` clause is executed. If none of the `case` clauses match, a `default` clause can be used to perform a default operation. For example:

[0076]

[0077] In this example, because the value of num is 2, the program will execute the code block following case2:. It should be understood that different programming languages ​​may have slight differences in the syntax and features of the switch statement, but the basic idea remains the same.

[0078] 7. Data Flow Analysis

[0079] Data flow analysis is an information analysis technique used to obtain information about the flow of data along the program execution path. The object of analysis is the data flow or possible values ​​along the program execution path. Simply put, data flow analysis is the process of analyzing the flow and usage of data within a program.

[0080] Data flow analysis can collect semantic information from program code and determine the definition and usage of variables at compile time or runtime using algebraic methods. At compile time, data flow analysis uses static analysis of program code to identify variable definition and usage points, assisting the compiler in compilation optimization and program verification. It can discover program behavior without actually running the program, thus helping developers understand the program. At runtime, data flow analysis dynamically monitors and analyzes data flow during program execution, detecting data status and changes in real time, providing strong support for program debugging, testing, and security protection.

[0081] Data flow analysis can involve several aspects, including: Definition-Use Chains: Tracing the definition and use points of variables to ensure that variables are correctly defined and initialized before use. Data Flow Diagrams: Visually illustrating the flow of data in a program to help developers intuitively understand the program structure.

[0082] 8. Instrumentation

[0083] Instrumentation refers to inserting probes (i.e., "stubs," also known as code segments that essentially collect information, such as function calls for collecting coverage information or performance data) into the source code of the program under test, while ensuring the original logical integrity of the program. The execution of these probes collects characteristic data of the program's operation (such as coverage and performance data). Analysis of this characteristic data reveals the data flow information covered by the program's execution.

[0084] 9. Stain Tracking

[0085] Taint tracking is a dynamic analytics technique used to trace the flow of sensitive data during program execution. By marking sensitive data within a program and tracking its propagation path, it can help uncover potential data breach risks.

[0086] Taint analysis technology abstracts data streams into triples, including tainted input sources, tainted sinks, and tainted data sanitizers. Taint tracking technology tracks tainted data input into the system to determine whether it has undergone sufficient cleansing, validation, filtering, or other sanitizing processes during its flow and propagation within the code, before directly performing sensitive operations that could pose security risks. One implementation of taint tracking technology involves inserting special taint propagation instructions into the native code to track the propagation path of sensitive data within the native code.

[0087] 10. Test Cases

[0088] A test case is a description of a testing task performed on a specific software product. Testing tasks can target aspects of the software product such as functionality, performance, security, compatibility, and usability.

[0089] For example, in functional testing, test cases can be used to verify whether various functions of a software product are correctly implemented according to the requirements specification. For instance, in login function testing, test cases might include different username and password combinations to verify the correctness and security of the login function. Image upload function testing: Uploading various image formats to verify the compatibility and error handling capabilities of the image upload function. Search function testing: Inputting various fuzzy keywords to test the accuracy and relevance of the search function. Payment function testing: Simulating various payment failure scenarios to test the exception handling and user guidance of the payment function.

[0090] For performance testing, test cases can be used to evaluate the software's efficiency and response to events under specific conditions. For example, by simulating a large number of users accessing the system simultaneously, the system's load capacity and response events can be tested to ensure that the system can still run stably under high concurrency.

[0091] For security testing, test cases can be used to check for security vulnerabilities and potential risks in software. For example, by simulating malicious functions and unauthorized access, the system's security capabilities and data protection mechanisms can be tested.

[0092] User acceptance testing (UAT) is conducted in the final stage of software development. Test cases are used to simulate the usage scenarios and needs of end users to verify whether the software meets user expectations and requirements.

[0093] In general, test cases can include inputs, expected outputs, and execution steps to help testers accurately execute and verify test results.

[0094] Figure 1 A schematic diagram of a test process provided for an embodiment of this application, such as Figure 1As shown, the testing process may include: inputting the source code of the software to be tested into the testing engine; providing an initial seed file as the original test cases; filtering and mutating the original test cases to obtain mutated test cases; inputting the mutated test cases into the executable program compiled from the source code and running it; obtaining test feedback information (such as code coverage) after execution; determining whether to retain the mutated test cases based on evaluation rules, for example, whether the code coverage has increased; if it has increased, it indicates that the test case is relatively useful, and the test case is added to the test case queue for retention. Then, test cases are filtered from the test case queue for mutation again, and this process is repeated, mutating, testing, and executing repeatedly until the program crashes. At this point, the input test cases are the test cases that the user wants to trigger the program vulnerability.

[0095] This application provides a guided testing system with a specific objective: to quickly cover the target point (vulnerability point) expected by the user. The guided testing system provided in this application provides guidance information for test cases, using the triggering target point as the mutation method. Based on this guidance information, the system guides the mutation process of test cases, causing them to mutate towards the target point. This reduces exploration on irrelevant paths that cannot reach the target point, and generates test cases that can reach the target point at a relatively fast speed.

[0096] Figure 2 This application provides a guided testing system 10. This system 10 can execute the test case generation method provided in this application. For example... Figure 2 As shown, the test system 10 may include a static analysis module 110, an information collection module 112, and a variation feedback module 113.

[0097] The static analysis module 110 can be used to perform semantic analysis on the source code of the software under test, identify program code in the source code that may contain vulnerabilities, and record them as target points.

[0098] The information collection module 112 can be used to process the source code according to the target point. The processed source code has the function of collecting the constraint information of test cases. Thus, the test cases can be sent into the executable program of the processed source code to run and obtain the constraint information of the test cases.

[0099] The mutation feedback module 113 can be used to guide the test case to make targeted mutations based on the constraint information of the test case, and output the mutated test case. Optionally, the mutation feedback module 113 can also output test feedback information for the test case, which is used to evaluate whether to retain the test case.

[0100] In one design, the static analysis module 110, information collection module 112, and mutation feedback module 113 can be hardware modules. In another design, the aforementioned static analysis module 110, information collection module 112, and mutation feedback module 113 can be software modules, and the test system 10 can implement the functions of these modules by executing the corresponding software modules through hardware.

[0101] For example, at the hardware level, system 10 may include processor 21 and memory 22. Processor 21 and memory 22 may be connected via bus 23.

[0102] Processor 21 is the computing and control core of test system 10. It can be a central processing unit (CPU) or other specific integrated circuits. Processor 210 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0103] The memory 22 is typically used to store computer program instructions and data. In one example, the memory 22 stores executable program code for the static analysis module 110, the information collection module 112, and the mutation feedback module 113. The processor 21 executes this executable program code to implement the functions of the static analysis module 110, the information collection module 112, and the mutation feedback module 113, respectively, thereby realizing the test case generation method provided in this application embodiment. That is, the memory 22 stores instructions for the test system 10 to execute the test case generation method provided in this application. Optionally, the data in the memory 22 may include the source code of the software to be tested. After processing the source code, the processor 21 can generate an executable program of the processed source code and store the executable program of the processed source code in the memory 22. The processor 21 can also execute the executable program to run test cases.

[0104] The memory 22 can be RAM or a hard disk. RAM has the advantage of fast access speed, and the processor 21 can interact directly with RAM. RAM may include dynamic random access memory (DRAM), static random access memory (SRAM), storage class memory (SCM), etc., and may also be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), etc. The hard disk is used for persistent storage of data and program code, and may be non-volatile memory, such as ROM, flash memory, hard disk drive (HDD), solid state drive (SSD), SCM, etc., without specific limitations. In this embodiment, the hard disk can be used to persistently store the program code of the aforementioned modules and the source code of the software to be tested. The processor 21 can read this program code from the hard disk into RAM.

[0105] In another possible design, one or more of the above modules are hardware modules, and the remaining modules are software modules. For example, the static analysis module 110 and the information collection module 112 are software modules, while the mutation feedback module 113 is a hardware module. This application does not limit this aspect. Furthermore, in each of the above possible designs, some modules may be the same module; for example, the static analysis module 110 and the information collection module 112 may be the same module. Additionally, Figure 2 The structure of the test system 10 shown is only an example; the test system 10 in actual applications may have more or fewer modules.

[0106] The following is applied to Figure 2 Taking the test system 10 shown as an example, the test case generation method provided in this application embodiment will be introduced. For ease of understanding, it will be described in two stages below. The first stage (see Figure 3 The second stage (see below): The process of generating the executable program (used to run test cases) of the software under test. Figures 4-6B ): Test case variation process.

[0107] Figure 3 This is a flowchart illustrating the method for generating an executable program for the software under test provided in this application embodiment. Figure 2Taking the test system 10 shown as an example, the method may include some or all of the following steps. For example... Figure 3 As shown, the method may include the following steps:

[0108] Step 301: Determine the target point in the source code of the software to be tested.

[0109] The static analysis module 110 acquires the source code of the software under test, for example in... Figure 2 In the process, the source code of the software to be tested is retrieved from memory 22. Alternatively, it may receive source code input by the user, etc., without specific limitations. Then, the static analysis module 110 determines the target points in the source code. This can be understood as a suspected or potential vulnerability point in the source code. Specifically, a target point may include a code segment in the source code that may contain vulnerabilities; this code segment may include one line of code or multiple lines of code. The following explanation uses a single line of code in the source code as the target point (referred to as a line of code) as an example.

[0110] The static analysis module 110 can determine target points in several ways. In one method, the target point is manually specified, such as by the source code developer or user. In another method, the static analysis module 110 obtains a static analysis report of the source code and determines the target point based on the report. The static analysis report indicates the target point; for example, it includes, but is not limited to, information indicating the target point's location, such as the line number of the target point, or the line of code corresponding to the target point. The static analysis report can be determined by the static analysis module 110 through static rule analysis of the source code, or it can be a static analysis report obtained by the static analysis module 110 from other devices. The number of target points indicated in the static analysis report can be one or more, without limitation. For ease of description, the following explanation uses a single target point as an example.

[0111] Step 302: Determine the relevant functions for the target point.

[0112] The static analysis module 110 can use data flow analysis methods to determine the relevant code of the target point. This process may include: determining the data flow associated with the target point; determining the relevant nodes of the data flow based on the data flow associated with the target point; and determining the functions related to the target point (target point-related functions) based on the relevant nodes of the data flow. The process is described in detail below:

[0113] The static analysis module 110 sends the source code and target point information of the software under test into the data flow analysis. By tracing the target point location backward in the source code, a data flow related to the target point can be obtained. This data flow indicates the variables related to the target point and the flow (relationship) between these variables. The lines of code containing operations that affect any variable in this data flow are designated as data flow-related nodes, and the functions containing these lines of code (i.e., data flow-related nodes) are designated as the related functions of the target point. The target point information includes, but is not limited to, one or more of the following: information indicating the location of the target point (such as line number) and / or the content of the target point (the program code included in the target point).

[0114] For example, a target point includes at least one variable, and the variables contained within the target point are strongly correlated variables. By performing reverse data flow tracing on the strongly correlated variables, a data flow and the operations that affect any variable in that data flow are obtained. Reverse tracing can be understood as working backward from the target point location in the source code to deduce the other variables that affect the strongly correlated variable.

[0115] This data flow indicates the direction of variable flow. The line of code containing an operation that affects any variable in this data flow is the data flow-related node. For example, if the target point includes variable 'a', and we trace back to the line of code: `a = b / / Assign the value of b to a`, then 'b' is a variable strongly related to 'a'. This assignment operation is an operation that affects the variable, and this line of code is a data flow-related node. Tracing further up, we get the line of code: `b = c / / Assign the value of c to b`, then 'c' is a variable strongly related to 'b'. This operation is also an operation that affects the variable, and this line of code is also a data flow-related node.

[0116] In summary, we can obtain a data flow: a←b←c. Here, a, b, and c are all variables related to the target point. The lines of code containing operations that affect the values ​​of a, b, and c are the data flow-related nodes, and the functions containing these lines of code are the target point-related functions.

[0117] Optionally, the relationship between data flow-related nodes can also be obtained based on the data flow: target point (including a) ← line of code (a = b) ← line of code (b = c).

[0118] Optionally, the static analysis module 110 can assign scores to data flow-related nodes based on the proximity of their relationships to the target point. Nodes closer to the target point receive higher scores, and vice versa. This distance can refer to the distance within the data flow relationship or the correlation between the target and target data flow nodes; closer nodes have higher correlation. For example, in the above example: target point (containing a) ← line of code (a=b) ← line of code (b=c), then the score of line of code (a=b) is higher than that of line of code (b=c). It should be noted that the distance within the data flow relationship may differ from the distance between lines of code in the source code. For instance, the target point's line number might be 100, the line number of line of code (a=b) might be 50, and the line number of line of code (b=c) might be 70. The scores of data flow-related nodes can be used as a factor in subsequently evaluating the value of test cases, which will be explained below and will not be repeated here.

[0119] Step 303: Based on the relevant functions of the target point, selectively instrument the source code and compile it to obtain the executable program of the software to be tested.

[0120] The information collection module 112 can insert probes (or "stubs") with specific functions into the source code. As mentioned earlier, a probe is essentially a piece of code, but it does not change the original logic and functionality of the source code. Compiling the source code containing the probe—that is, compiling the probe and the source code together—results in executable binary code and the executable program of the software under test. This executable program can run test cases, and during execution, the probe will collect the program execution status of the test cases.

[0121] Selective instrumentation refers to the ability of the information gathering module 112 to insert probes into relevant functions of the target point, rather than requiring probes to be inserted into all functions of the source code. The following section provides a detailed explanation of the probe functionality:

[0122] 1. Constraint probe, used to collect constraint information.

[0123] The information collection module 112 inserts constraint probes (code) into the relevant functions of the target point. When these probes run, they will collect information about the constraints traversed by the test cases in the execution path of the relevant functions of the target point.

[0124] For example, the information collection module 112 can insert constraint probe code before all branch jump instructions of the relevant functions of the target point. Branch jumps include, but are not limited to: conditional jumps (such as jumps in switch statements, if statements, for and while loops), function calls and returns, and unconditional jumps (such as goto statements). When test cases are input into the executable program, these probes are executed and can collect information about the corresponding constraints. The constraint information indicates the execution status of the test cases on the constraints, such as whether the test case executed the constraint, which branch of the constraint was selected, and the attribute information of the constraint, etc. Among them, the attribute information of the constraint may include, but is not limited to: one or more elements such as the shape, length, keyword, constraint type (if or switch, etc.), whether the constraint contains constants, and the type of the variables contained in the constraint. For example, if(int a = 3), the type of this constraint is if, the constraint contains variable a, the length of variable a is xxx bytes (depending on the byte length of the int type), and 3 is a constant (which can be regarded as a keyword).

[0125] 2. Taint tracking probes are used to collect the propagation path of bytes included in test cases in the source code.

[0126] Information collection module 112 can insert taint tracking probes (code, such as taint propagation instructions) into the relevant functions of the target point. When these probes run, they will collect the propagation path of the bytes (taint data) included in the test case in the source code. The propagation path indicates which lines of code the flow enters.

[0127] 3. Coverage probes are used to collect data on how well test cases cover the code.

[0128] The information collection module 112 can insert coverage probes (code) into the relevant functions of the target point. When these probes run, they will collect the code paths covered by the test cases in the relevant functions of the target point, that is, which lines of code were executed and the coverage rate. Coverage can refer to the ratio of the number of lines of code executed by the first test case to the total number of lines of code included in the source code.

[0129] The information collection module 112 can insert probes with one or more functions within the relevant functions of the target point. The insertion positions of probes with different functions within the functions may be the same or different. Then, the source code with the probes inserted is compiled to obtain the executable program of the software to be tested. After obtaining the executable program, test cases can be run using it. It is worth noting that the executable program obtained here does not need to be regenerated and can be reused.

[0130] The above design, through selective instrumentation in the source code, reduces the collection and use of irrelevant information during testing. Existing large, commonly used open-source C / C++ libraries such as libming and binutils have tens of thousands of lines of source code and contain hundreds or thousands of functions. Taking CVE-2018-17360 as an example, reaching the target point involves over 100 paths and over 20,000 constraints. Conventional solutions spend too much time exploring irrelevant paths or resolving irrelevant constraints in the generated input, slowing down the generation of valid input and even the speed of test cases that trigger crashes at specific target points.

[0131] Figure 4 This is a flowchart illustrating a test case generation method provided in an embodiment of this application. The method is described as follows: Figure 2 Taking the test system 10 shown as an example, the method may include some or all of the following steps. For example... Figure 4 As shown, the method may include the following steps:

[0132] Step 401: Obtain the test cases for the software to be tested (denoted as the first test case).

[0133] Step 402, input the first test case into the aforementioned ( Figure 3 The method embodiment obtains the constraint information of the first test case by running the executable program of the software to be tested.

[0134] The constraint information for the first test case may include information collected based on one or more probes.

[0135] For example, constraint information includes constraint information collected by constraint probes, that is, information about the constraints traversed by the first test case in the execution path of the relevant function at the target point. The constraint information can indicate the execution status of the first test case on the traversed constraints and the attribute information of the constraints, etc.

[0136] In addition, constraint information can also include the association between the bytes included in the first test case and the constraints. This association can be determined based on the byte propagation paths collected by the taint tracking probe. A test case can be viewed as data composed of many bytes. The taint tracking probe can tag the bytes included in the test case, record the byte propagation paths, thereby determining which bytes affect the constraints and further obtaining the correspondence between constraints and bytes. Based on these correspondences, test cases can be guided to selectively mutate these bytes.

[0137] Step 403: Determine the priority order of constraints.

[0138] The information collection module 112 can first determine the priority of the new constraints in the first test case, then obtain the priority information of the historical constraints, and sort the new constraints and historical constraints according to this priority.

[0139] In one example, constraint priority is measured by a score. The constraint score depends on the score of the test case from which it originates, and the test case score is the sum of the scores of all data flow-related nodes covered by that test case after its execution. It should be noted that since test cases are constantly evolving, new and old test cases may contain the same constraints. Here, we can focus only on the priority of newly added constraints; that is, new test cases may have new constraints, and the score of the new constraint comes from the score of this new test case, which is based on the sum of the scores of all data flow-related nodes covered by that test case.

[0140] For example, suppose the historical constraints include constraint A and constraint B, constraint A has a score of 40 points and constraint B has a score of 50 points. Constraint A may come from test case A and constraint B may come from test case B.

[0141] Assume the relationship between the nodes in the data stream is as follows: Target 109 ← Node 100 (70) ← Node 98 (60) ← Node 97 (50) ← Node 90 (40) ← Node 77 (30) ← Node 67 (20) ← Node 66 (10). The number after a node represents the line number of the code corresponding to that node, and the value in parentheses after a node represents the score of that node.

[0142] Assume that based on the constraint information of the first test case, it is determined that the first test case executed constraints A, B, and C, where constraint C is a newly added constraint. If the first test case covers nodes 66, 67, and 77, then the score for the first test case is 10 + 20 + 30 = 60 points, thus the score for constraint C is 60. The data flow-related nodes covered by the first test case can be determined based on the coverage rate or constraint information of the first test case.

[0143] Based on the scores of each constraint, the priority order of the constraints can be determined as follows: constraint C > constraint B > constraint A.

[0144] Step 404: Determine the mutation strategy (denoted as the first mutation strategy) based on the priority ranking and the constraint information of the first test case, and guide the first test case to mutate based on the first mutation strategy to obtain the mutated test case (denoted as the second test case).

[0145] Priority ranking can be used to determine whether to retain test cases, select test cases for mutation, and so on. The constraint with the highest priority is the one whose test case is selected for mutation. For example, in the example above, constraint C has the highest priority, so the first test case is selected for mutation. If constraint B has the highest priority, then test case B is selected for mutation. It should be understood that the test case mutation process is an ongoing exploration process. Many test cases may be generated before a test case that triggers the target point is obtained. Therefore, the test cases used for mutation can be dynamically adjusted based on priority ranking.

[0146] This section uses constraint C, which has the highest priority in the first test case, as an example. Furthermore, constraints with higher priority can be resolved first during subsequent mutations.

[0147] Based on the constraint information of the first test case, the constraints to be resolved in the first test case can be identified. For example, suppose constraint C includes two branches: branch 1 and branch 2. Suppose that based on the constraint execution status, it is determined that the first test case executed branch 1 of constraint C, then branch 2 is the constraint to be resolved. Then, the execution conditions of branch 2 can be determined based on the attribute information of constraint C; the bytes associated with branch 2 can be determined based on the association between the bytes included in the first test case and the constraints; and the resolution strategy / mutation strategy (denoted as the first mutation strategy) can be determined based on the execution conditions of branch 2 and the bytes associated with branch 2.

[0148] For example, solution strategies may include, but are not limited to, direct copying and length exploration. These solution strategies can be pre-defined or generated based on the attribute information of the constraint to be solved. For example, suppose the execution condition of branch 2 in constraint C is if(int a=3), and the input byte corresponding to constraint C is determined to be a based on the correspondence between bytes and constraints, then the solution strategy could be to directly copy the input byte a=3.

[0149] For example, the first mutation strategy includes a mutation object and / or a mutation method; the bytes associated with the constraint to be resolved in the first test case, and the mutation method indicating the operation that satisfies the constraint to be resolved by the mutation object. For example, in the example above, the mutation object includes the bytes associated with branch 2, and the mutation method includes the operation that satisfies the execution conditions of branch 2. Based on this first mutation strategy, the first test case is mutated, such as by specifically mutating the bytes associated with branch 2 in the first test case according to the indicated operation, to obtain the second test case.

[0150] In one possible scenario, the input byte a of the second test case is 3. If the second test case is input into the aforementioned instrumented executable program and run, the second test case can execute to constraint C and select branch 2 of constraint C to continue execution. At this time, the second test case may cover more data flow related nodes, be closer to the target point or reach the target point, reduce exploration on irrelevant paths, thereby accelerating the generation of test cases that can trigger the target point and improving the mutation efficiency of the test case generation process.

[0151] Figure 5 This is a flowchart illustrating another test case generation method provided in this application embodiment. The method is as follows: Figure 2 Taking the test system 10 shown as an example, the method may include some or all of the following steps. For example... Figure 5 As shown, the method may include the following steps:

[0152] Step 501: Input the second test case into the executable program of the software under test and run it to obtain the execution information of the second test case. Refer to the relevant description of step 402 above, which will not be repeated here.

[0153] It should be noted that the executable program here can be... Figure 3 The executable program obtained from the method implementation may not be otherwise. Since this method does not require constraint information, the difference lies in the fact that the executable program used here may not include constraint probes, but it must include taint tracking probes. For example, these taint tracking probes can be placed before the line of code where the target point is located. These probes can collect the association between the bytes included in the test cases and the target point. For more details, please refer to [link to relevant documentation]. Figure 3 The description of the method embodiments is omitted here. In practical applications, for the sake of brevity, all other methods can be used. Figure 3 The executable program obtained from the method embodiment. Therefore, the execution information of the second test case may be the constraint information of the second test case, or it may only include the association between the bytes included in the second test case and the target point.

[0154] Step 502: Determine the vulnerability type and / or relevant variables of the target point.

[0155] The vulnerability type of the target point indicates what kind of vulnerability may exist at the target point. Vulnerability types include, but are not limited to: buffer overflow, integer overflow, null pointer dereference, and use after free.

[0156] The methods for determining the vulnerability type of a target point can include: Method 1, determining it based on the static analysis report of the first test case, where the static analysis report may include the vulnerability type of the target point. Method 2, performing semantic analysis on the source code and / or additional information of the target point to obtain the vulnerability type of the target point. The additional information of the target point includes, but is not limited to, one or more of the following: information indicating the location of the target point (such as line numbers), the content of the target point (the program code included in the target point), and the vulnerability type (such as the vulnerability type of the target point indicated in the static analysis report). Semantic analysis refers to analyzing a piece of content and extracting the target content. In the embodiments of this application, semantic analysis can be implemented using methods such as manual or expert experience, neural network models, or data flow analysis, and is not specifically limited.

[0157] The relevant variables for the target point can be found in the description of step 302, and will not be repeated here. These relevant variables can be determined using data flow analysis or semantic analysis. Semantic analysis extracts the context of the target point, which may include code preceding the target point in the source code. By tracing back based on the target point's context and the variables it includes, the relevant variables for the target point can be obtained, as described above.

[0158] Semantic analysis methods can improve the vulnerability verification efficiency of existing testing engines.

[0159] Step 503: Determine the mutation strategy (denoted as the second mutation strategy) based on the vulnerability type and related variables of the target point, and guide the second test case to mutate based on the second mutation strategy to obtain the mutated test case (denoted as the third test case).

[0160] It's understandable that resolving constraints is to reach the target point as quickly as possible. Once the target point is reached, mutations can be performed according to the mutation strategies set for different vulnerability types.

[0161] Assuming the second test case has reached the target point, a second mutation strategy can be determined based on the vulnerability type and / or relevant variables of the target point. The second mutation strategy indicates the mutation object and / or mutation method. The mutation object includes bytes associated with the variables related to the target point, and the mutation method indicates the operation that satisfies the vulnerability triggering conditions of the target point. For example, assuming the vulnerability type of the target point is an integer overflow, an integer overflow vulnerability can be resolved by increasing the numerical value, and a buffer overflow vulnerability can be resolved by increasing the overflowed buffer data. Constraint-related bytes may affect the relevant variables of the target point; therefore, guiding the mutation of bytes associated with the variables of the target point based on the second mutation strategy can solve the problem of test cases failing to trigger the vulnerability even after reaching the target point.

[0162] It should be noted that, Figure 4 Method Implementation Examples and Figure 5 The method implementation examples can be used individually, such as when only one method is used during the test case variation process. Figure 4 The method shown, or only using Figure 5 The method shown can also be used. Figure 4 The method shown and Figure 5 The methods shown are used in combination. For example, in one example, first execute... Figure 4 Execute after the method flow Figure 5 The method and process. For example, Figure 6A An example of combining them is provided. Figure 6B This is a specific example of a combined approach. Figure 6A and Figure 6B In one testing approach, the code under test and the relevant descriptions of the target point are first fed into data flow analysis. Reverse data flow analysis of the target point yields information about its data flow-related functions and nodes. Selective instrumentation of data flow-related functions in the code under test results in instrumented executable binary code. Fuzzing then begins, from initial test cases to running the executable program. During continuous execution, taint analysis techniques are used to establish the relationship between input bytes and code path constraints, prioritizing these constraints based on data flow-related node scoring information. Simultaneously, semantic analysis is performed on the code under test and the target point. Semantic analysis of additional information about the target point reveals its vulnerability type, and reverse data flow tracing of the target point's context yields its key variables. Based on the priority order of path constraints, the corresponding solution strategy is determined to resolve the corresponding path constraints. Furthermore, the vulnerability types and key variables obtained from semantic analysis are used to provide targeted guidance for mutation generation test cases, enabling efficient verification within a short period of time (e.g., 48 hours) to obtain specific crash input test cases (Proof of Concept, PoC) and corresponding fuzzing (Fuzz Testing, fuzz) reports.

[0163] Based on the same inventive concept as the method embodiments, this application also provides a computing device for performing the above-described... Figures 3-6B The method of any method embodiment. For example... Figure 7 As shown, in one example, the computing device 700 includes an acquisition module 701, a testing module 702, and a generation module 703. Optionally, it also includes a determination module 704. Specifically, in the computing device 700, the modules are connected to each other through a communication path.

[0164] Module 701 is used to obtain the first test case; see also... Figure 4 The relevant description of step 401 in the method embodiment will not be repeated here.

[0165] Test module 702 is used to insert the first test case input into the software under test equipped with a probe and run it to obtain constraint information of the first test case collected by the probe. The constraint information indicates the execution status of constraints traversed by the first test case in the critical path. The critical path includes the execution path of code related to a target point, and the target point includes lines of code in the source code of the software under test that contain potential vulnerabilities. (See also...) Figure 4 For a description of step 402 in the method embodiments, or see the following: Figure 5 The relevant description of step 501 in the method embodiment will not be repeated here.

[0166] The generation module 703 is used to generate a second test case based on the first test case and its constraint information, wherein the constraint information indicates the variation mode of the first test case; the second test case is used to test the software under test. (See also...) Figure 4 For the relevant descriptions of steps 403 and 404 in the method embodiments, or refer to... Figure 5 The relevant description of step 503 in the method embodiment will not be repeated here.

[0167] In one possible implementation, the determining module 704 is used to determine the functions in the source code related to the target point;

[0168] The probe is inserted into the source code based on the function described above;

[0169] When the test module 702 is run in the software under test after the first test case input is inserted into the probe, it is specifically used for:

[0170] The first test case is input into the executable program obtained by compiling the source code with the probe inserted and then running it.

[0171] In one possible implementation, when determining the function in the source code related to the target point, the determining module 704 is specifically used for:

[0172] A data flow is determined based on the source code and the target point, the data flow indicating variables related to the target point and the relationships between the variables; data flow-related nodes are determined based on the data flow; the data flow-related nodes include the lines of code in the source code where operations affecting the variables are located; the functions related to the target point include the functions containing the lines of code.

[0173] In one possible implementation, the probe is further configured to collect coverage information of the first test case, the coverage information indicating the lines of code covered by the first test case;

[0174] The acquisition module 701 is further configured to acquire information about historical constraints, wherein the information about historical constraints indicates the priority of the historical constraints;

[0175] The determining module 704 is further configured to determine new constraints in the first test case based on the constraint information of the first test case; determine the priority of the new constraints, wherein the priority of the new constraints is determined based on the coverage information of the first test case and the score of the data flow related nodes; the higher the correlation between the data flow related nodes and the target point, the higher the score; and the priority of the new constraints is higher than the priority of the historical constraints.

[0176] In one possible implementation, the constraint information includes the association between the bytes included in the first test case and the constraints traversed in the key path;

[0177] When generating a second test case based on the first test case and its constraint information, the generation module 703 is specifically used for:

[0178] A first mutation strategy is determined based on the constraint information of the first test case and the association relationship. The first mutation strategy indicates the mutation object and / or mutation method. The mutation object includes the bytes associated with the constraint to be resolved in the first test case, and the mutation method indicates the condition that the mutation object satisfies the constraint to be resolved.

[0179] The first test case is mutated according to the first mutation strategy to generate the second test case.

[0180] In one possible implementation, the probe is further configured to collect the propagation path of the bytes included in the first test case within the lines of code covered by the first test case; the association is determined based on the propagation path.

[0181] In one possible implementation, the acquisition module 701 is further configured to determine the description information of the target point, the description information including the vulnerability type of the target point and / or variables related to the target point;

[0182] The generation module 703 is further configured to: generate a second test case based on the first test case and the description information of the target point.

[0183] In one possible implementation, the probe is also used to collect the propagation path of the bytes included in the first test case in the lines of code covered by the first test case;

[0184] When generating a second test case based on the first test case and the description information of the target point, the generation module 703 is specifically configured to: determine a second mutation strategy based on the description information of the target point, wherein the second mutation strategy indicates the mutation object and / or mutation method; the mutation object includes bytes associated with variables related to the target point, and the mutation method indicates the operation of the mutation object that satisfies the vulnerability triggering condition of the target point; and mutate the first test case according to the first mutation strategy to generate the second test case.

[0185] In one possible implementation, when acquiring the description information of the target point, the acquisition module 701 is specifically used for:

[0186] Based on the source code, determine the vulnerability type of the target point and / or variables related to the target point; or...

[0187] Obtain static analysis data of the source code, the static analysis data including the vulnerability type of the target point and / or variables related to the target point.

[0188] For example, the implementation of the generation module 703 in the computing device 700 will be described below. Similarly, the implementation of the acquisition module 801 and the update module 804 can refer to the implementation of the generation module 703.

[0189] When implemented in software, the generation module 703 can be an application or code block running on a computer device. The computer device can be at least one of a physical host, virtual machine, container, or other computing device. Furthermore, there can be one or more computer devices. For example, the generation module 703 can be an application running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application can be distributed within the same availability zone (AZ) or in different AZs. Similarly, the multiple hosts / virtual machines / containers used to run the application can be distributed within the same region or in different regions. Typically, a region can include multiple AZs.

[0190] Similarly, multiple hosts / virtual machines / containers used to run the application can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a region can include multiple VPCs, and a VPC can include multiple Availability Zones (AZs).

[0191] When implemented in hardware, the generation module 703 may include at least one computing device, such as a server. Alternatively, the generation module 703 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0192] The multiple computing devices included in the generation module 703 can be distributed within the same Availability Zone (AZ) or in different AZs. Similarly, the multiple computing devices included in the generation module 703 can be distributed within the same region or in different regions. Likewise, the multiple computing devices included in the generation module 703 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0193] It should be noted that the module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. The functional modules in this embodiment can be integrated into one module, or each module can exist physically separately, or two or more modules can be integrated into one module. For example, the acquisition module 701 and the determination module 704 can be integrated into one module, or the acquisition module 701 and the test module 702 can be the same module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0194] This application also provides a computing device 800. For example... Figure 8As shown, the computing device 800 includes a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate with each other via the bus 802. The computing device 800 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 800.

[0195] The 802 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus 802 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 802 may include a path for transmitting information between various components of the computing device 800 (e.g., memory 806, processor 804, communication interface 808).

[0196] Processor 804 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0197] The memory 806 may include volatile memory, such as random access memory (RAM). The processor 804 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0198] The memory 806 stores executable program code, and the processor 804 executes the executable program code to implement the functions of the aforementioned acquisition module 701, test module 702, generation module 703, and determination module 704, thereby realizing the test case generation method. That is, the memory 806 stores instructions for the computing device 700 to execute the test case generation method provided in this application.

[0199] The communication interface 808 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 800 and other devices or communication networks.

[0200] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server. In some embodiments, the computing device may also be a desktop computer, a laptop computer, or a smartphone, or other terminal device.

[0201] like Figure 9 As shown, the computing device cluster includes at least one computing device 800. The memory 806 of one or more computing devices 800 in the computing device cluster may store the same instructions for executing test case generation methods.

[0202] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also store partial instructions for executing the test case generation method. In other words, a combination of one or more computing devices 800 can jointly execute the instructions for executing the test case generation method.

[0203] It should be noted that the memory 806 in different computing devices 800 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the data access device. That is, the instructions stored in the memory 806 of different computing devices 800 can implement the functions of one or more modules among the acquisition module 701, test module 702, generation module 703, and determination module 704.

[0204] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 10 One possible implementation is shown. For example... Figure 10 As shown, two computing devices 800A and 800B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 806 in computing device 800A stores instructions for executing the functions of the acquisition module 701 and the determination module 704. Simultaneously, the memory 806 in computing device 800B stores instructions for executing the functions of the test module 702 and the generation module 703.

[0205] It should be understood that Figure 10 The functions of the computing device 800A shown can also be performed by multiple computing devices 800. Similarly, the functions of the computing device 800B can also be performed by multiple computing devices 800.

[0206] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 9 and Figure 10 The connection method of the computing device cluster is different in that the memory 806 of one or more computing devices 800 in the computing device cluster can store the same instructions for executing the test case generation method.

[0207] In some possible implementations, the memory 806 of one or more computing devices 800 in the computing device cluster may also store partial instructions for executing the test case generation method. In other words, a combination of one or more computing devices 800 can jointly execute the instructions for executing the test case generation method.

[0208] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute a test case generation method, or instructs the computing device to execute a test case generation method.

[0209] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute a test case generation method, or instruct the computing device to execute a test case generation method.

[0210] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0211] The software implementation portion of the above embodiments can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0212] The various illustrative logic units and circuits described in the embodiments of this application can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0213] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, software units executed by a processor, or a combination of both. The software units can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC.

[0214] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0215] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative descriptions of the application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Thus, if such modifications and modifications fall within the scope of the claims and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A test case generation method, characterized in that, The method includes: Obtain the first test case; The first test case is input and run in the software under test with a probe to obtain the constraint information of the first test case collected by the probe. The constraint information indicates the execution status of the constraints traversed by the first test case in the execution path of the code related to the target point. The target point includes the line of code in the source code of the software under test that has a potential vulnerability. A second test case is generated based on the first test case and its constraint information, wherein the constraint information is used to indicate the variation mode of the first test case; the second test case is used to test the software to be tested.

2. The method as described in claim 1, characterized in that, The method further includes: Identify the functions in the source code that are related to the target point; The probe is inserted into the source code based on the function described above; The first test case is input and run in the software under test after the probe is inserted, including: The first test case is input into the executable program obtained by compiling the source code with the probe inserted and then running it.

3. The method as described in claim 2, characterized in that, Determine the functions in the source code related to the target point, including: A data flow is determined based on the source code and the target point, the data flow indicating variables related to the target point and the relationships between the variables; The data flow-related nodes are determined based on the data flow; the data flow-related nodes include the lines of code in the source code where the operations affecting the variables are located; the functions related to the target point include the functions containing the lines of code.

4. The method as described in claim 3, characterized in that, The probe is also used to collect coverage information of the first test case, the coverage information indicating the lines of code covered by the first test case; The method further includes: Obtain information about historical constraints, which indicates the priority of the historical constraints; The new constraints in the first test case are determined based on the constraint information of the first test case; The priority of the newly added constraint is determined based on the coverage information of the first test case and the score of the data flow related node; the higher the correlation between the data flow related node and the target point, the higher the score; the priority of the newly added constraint is higher than the priority of the historical constraint.

5. The method according to any one of claims 1-4, characterized in that, The constraint information includes the association between the bytes included in the first test case and the constraints traversed in the key path; The generation of the second test case based on the first test case and its constraint information includes: A first mutation strategy is determined based on the constraint information of the first test case and the association relationship. The first mutation strategy indicates the mutation object and / or mutation method. The mutation object includes the bytes associated with the constraint to be resolved in the first test case, and the mutation method indicates the condition that the mutation object satisfies the constraint to be resolved. The first test case is mutated according to the first mutation strategy to generate the second test case.

6. The method as described in claim 5, characterized in that, The probe is also used to collect the propagation path of the bytes included in the first test case in the lines of code covered by the first test case; the association is determined based on the propagation path.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Obtain descriptive information of the target point, the descriptive information including the vulnerability type of the target point and / or variables related to the target point; A second test case is generated based on the first test case and the description information of the target point.

8. The method as described in claim 7, characterized in that, The probe is also used to collect the propagation path of the bytes included in the first test case in the lines of code covered by the first test case; Generate a second test case based on the first test case and the description information of the target point, including: Based on the description information of the target point, a second mutation strategy is determined, the second mutation strategy indicating the mutation object and / or mutation method; the mutation object includes bytes associated with variables related to the target point, and the mutation method indicates the operation of the mutation object that satisfies the vulnerability triggering condition of the target point; The first test case is mutated according to the first mutation strategy to generate the second test case.

9. The method as described in claim 7 or 8, characterized in that, The step of obtaining the description information of the target point includes: Based on the source code, determine the vulnerability type of the target point and / or variables related to the target point; or... Obtain static analysis data of the source code, the static analysis data including the vulnerability type of the target point and / or variables related to the target point.

10. A test case generation device, characterized in that, The device includes: The acquisition module is used to acquire the first test case; The testing module is used to insert the first test case input into the software under test with a probe and run it to obtain the constraint information of the first test case collected by the probe. The constraint information indicates the execution status of the constraints traversed by the first test case in the critical path. The critical path includes the execution path of the code related to the target point. The target point includes the line of code in the source code of the software under test that has a potential vulnerability. A generation module is used to generate a second test case based on the first test case and the constraint information of the first test case, wherein the constraint information is used to indicate the variation mode of the first test case; the second test case is used to test the software to be tested.

11. The apparatus as claimed in claim 10, characterized in that, The device also includes a determining module; The determining module is used to determine the functions in the source code related to the target point; The probe is inserted into the source code based on the function described above; When the testing module runs in the software under test after the first test case input is inserted into the probe, it is specifically used for: The first test case is input into the executable program obtained by compiling the source code with the probe inserted and then running it.

12. The apparatus as claimed in claim 11, characterized in that, When determining the functions in the source code related to the target point, the determining module is specifically used for: A data flow is determined based on the source code and the target point, the data flow indicating variables related to the target point and the relationships between the variables; data flow-related nodes are determined based on the data flow; the data flow-related nodes include the lines of code in the source code where operations affecting the variables are located; the functions related to the target point include the functions containing the lines of code.

13. The apparatus as claimed in claim 12, characterized in that, The probe is also used to collect coverage information of the first test case, the coverage information indicating the lines of code covered by the first test case; The acquisition module is also used to acquire information about historical constraints, the information about historical constraints indicating the priority of the historical constraints; The determining module is further configured to determine new constraints in the first test case based on the constraint information of the first test case; determine the priority of the new constraints, wherein the priority of the new constraints is determined based on the coverage information of the first test case and the score of the data flow related nodes; the higher the correlation between the data flow related nodes and the target point, the higher the score; and the priority of the new constraints is higher than the priority of the historical constraints.

14. The apparatus according to any one of claims 10-13, characterized in that, The constraint information includes the association between the bytes included in the first test case and the constraints traversed in the key path; When generating a second test case based on the first test case and its constraint information, the generation module is specifically used for: A first mutation strategy is determined based on the constraint information of the first test case and the association relationship. The first mutation strategy indicates the mutation object and / or mutation method. The mutation object includes the bytes associated with the constraint to be resolved in the first test case, and the mutation method indicates the condition that the mutation object satisfies the constraint to be resolved. The first test case is mutated according to the first mutation strategy to generate the second test case.

15. The apparatus as claimed in claim 14, characterized in that, The probe is also used to collect the propagation path of the bytes included in the first test case in the lines of code covered by the first test case; the association is determined based on the propagation path.

16. The apparatus according to any one of claims 10-15, characterized in that, The acquisition module is further configured to determine the description information of the target point, the description information including the vulnerability type of the target point and / or variables related to the target point; The generation module is further configured to: generate a second test case based on the first test case and the description information of the target point.

17. The apparatus as claimed in claim 16, characterized in that, The probe is also used to collect the propagation path of the bytes included in the first test case in the lines of code covered by the first test case; When generating a second test case based on the first test case and the description information of the target point, the generation module is specifically used to: determine a second mutation strategy based on the description information of the target point, wherein the second mutation strategy indicates the mutation object and / or mutation method; the mutation object includes bytes associated with variables related to the target point, and the mutation method indicates the operation of the mutation object that satisfies the vulnerability triggering condition of the target point; and mutate the first test case according to the first mutation strategy to generate the second test case.

18. The apparatus as claimed in claim 16 or 17, characterized in that, When acquiring the description information of the target point, the acquisition module is specifically used for: Based on the source code, determine the vulnerability type of the target point and / or variables related to the target point; or... Obtain static analysis data of the source code, the static analysis data including the vulnerability type of the target point and / or variables related to the target point.

19. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 9.

20. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that, Includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 9.

Images