A software testing method and device based on target drive
Through the target-driven Concolic Testing technology, the expansion unit is built using function call relationships and potential errors are verified, solving the problems of unit test false positives and Concolic test false negatives, achieving more efficient and accurate software testing.
Patent Information
- Application Number
- CN202110479754.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-04-30
AI Technical Summary
Unit tests have high false positive rates and are difficult to use directly in real software. Concolic tests have missed reports due to path explosion problems, resulting in a lot of manual work for testers to confirm potential errors.
The target-driven Concolic Testing (TDCT) technology is used to build an extension unit through the calculation of the correlation between the call relationships between functions, and potential errors are discovered using unit tests, and the target-driven search strategy is used at the system level to verify whether the error exists.
降低了单元测试的误报率,提高了测试的准确度和效率,减少了测试人员的手动工作量,提高了代码覆盖率并减少了漏报。
Smart Images

Figure CN113297069B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of software security testing, and in particular relates to a software testing method and device based on target driving. Background Art
[0002] Unit testing is a software testing method by which each unit of source code is tested to determine whether they are suitable for use. A unit is the smallest testable software component, usually performing a single cohesive function. Some people think that any test that goes out of a function is not a unit test. In fact, the definition of "unit" depends on yourself. If you are using functional programming, a unit is most likely a function. Your unit test will call this function with different parameters and assert whether it returns the expected result; in object-oriented languages, anything from a single method to an entire class can be a unit. Unit testing refers to checking and verifying this smallest testable component.
[0003] Unit testing can test each unit of the source code to determine their usability and security. Unit testing is a necessary step to ensure product quality, and developers can use it to find and correct defects in the source code. If unit testing is performed effectively, it may achieve significant benefits in terms of improving product quality, reducing development time / cost, optimizing source code design, and improving maintenance difficulty. Usually, unit testing requires testers to establish a test driver and use the driver to call the unit under test, which checks the returned results based on known values or the results of the previous execution of the same test (regression testing).
[0004] The weakness of unit testing is that because unit testing uses approximate symbolic input and return values to represent the context of the unit, most of the errors found in the unit are false alarms caused by infeasible paths, which means that these false alarms do not exist at the system level. The definition of "unit" affects the false alarm rate. For example, when a function is used as a unit, the false alarm rate is higher than when a layer of function calls is added around the function, and the false alarm rate decreases with the added functions until the entire program is used as a unit in the extreme case. At this time, the false alarm rate is 0, but as the scope of the unit expands, the program path will become more complex and the call depth will continue to increase, which makes the dynamic unit test performance continue to decrease. In addition, since unit testing has a large number of potential error reports, it is necessary to reverse the local path information to the global to confirm whether each error in these reports exists in the complete program, but it is difficult to confirm potential errors, which often brings a lot of manual work to test engineers and takes a lot of time.
[0005] Concolic testing is a form of symbolic execution, which has become an effective software testing method. The essence of concolic testing is to run the program with symbolic values instead of manually or randomly constructed inputs while executing the program concretely. Symbolic execution runs the code on symbolic inputs that are initially allowed to be arbitrary values, and collects branch conditions for the path on the execution path, that is, when the branch of program execution is based on symbolic values, the concolic execution engine tracks the two branches, maintains a set of path constraints on each branch, and uses a symbolic solver to find a satisfiable solution to the path constraints, obtains the concrete value of the current path to generate a test case for the path. Running the program again with the generated test case, it can follow the same path and hit the same error. Carefully selecting new branch paths can help testers systematically explore execution paths of programs with high code coverage without generating redundancy.
[0006] Unfortunately, concolic testing suffers from the well-known path explosion problem, since the number of different execution paths in a program is often exponential to the number of conditional statements, or even infinite in the worst case. Although symbolic execution tends to explore orders of magnitude more input paths than traditional testing, it often struggles to exhaust all interesting paths. Furthermore, due to the complexity of a program, it is often impossible to reach the deepest code inside a program. Even if a tool successfully reaches deep code, it will only consider input values that satisfy the few paths that can reach this code.
[0007] In summary, the problems and shortcomings of the prior art are:
[0008] The effective execution of unit testing may achieve significant benefits in terms of improving product quality, reducing development time / cost, optimizing source code design, and reducing maintenance difficulty. However, the results of unit testing are difficult to use directly in real software, and it still requires field engineers to manually generate the necessary global inputs for each target unit. Automated unit testing can perform a thorough analysis of smaller functions in the code, but there is the fact that certain restrictions on unit inputs or correlations between inputs that occur at the system level are not considered. Therefore, they usually bring a lot of false positives. Some of the latest research reduces false positives by filtering potential errors found in unit testing, but the confirmation of potential errors still brings a lot of workload for test engineers.
[0009] Concolic testing is widely used in unit testing due to its high code coverage and avoidance of generating redundant inputs on the same path. However, concolic testing suffers from path explosion, which is common in symbolic execution. Therefore, although this method has no false positives, it has a large number of false negatives. Although the importance of comprehensive software testing has been recognized, it requires a lot of work for testers to come up with relevant test cases, which leads to it rarely being put into practice in real software. Summary of the invention
[0010] The present invention develops a target-driven concolic testing (TDCT) technique by combining unit testing and concolic testing. TDCT reduces false positives in unit testing by constructing extended units based on the correlation between functions. At the same time, it automatically confirms potential errors found in unit testing and generates system-level inputs by adopting a target-driven search strategy in system-level concolic testing.
[0011] The technical solution adopted by the present invention is as follows:
[0012] A software testing method based on goal-driven includes the following steps:
[0013] Collect the calling relationships between functions, use the calling relationships between functions to calculate the correlation of functions, and build an expansion unit based on the correlation of functions;
[0014] Use unit tests to find potential bugs in your extension units;
[0015] Use a goal-driven search strategy to verify whether potential errors exist globally.
[0016] Furthermore, collecting the calling relationships between functions and calculating the correlation of functions by using the calling relationships between functions includes:
[0017] Concolic execution starts with the target program to be tested and a set of system-level test initial values as input, and the time threshold is set according to the actual situation; the concolic engine aims to obtain as many system-level paths as possible and stops running once the set time threshold is reached; at the same time, the calling relationship of functions on the running path is collected, and the calling frequency between functions is used to determine the correlation between functions by calculating the conditional probability.
[0018] Furthermore, after obtaining the calling frequencies between the functions, the correlation R between the two functions x and y is calculated by the following formula:
[0019]
[0020] Among them, F(x,y) represents the sum of the number of times function x calls function y and the number of times function y calls function x, F(x) represents the number of times function x appears in all paths, and F(y) represents the number of times function y appears in all paths.
[0021] Furthermore, the construction of the extension unit through function correlation includes: presetting a function correlation threshold according to actual conditions, and selecting related functions that meet the requirements according to the function correlation threshold to construct the extension unit; for each target function f, constructing a set of extension units, including f itself, a caller e that meets the function correlation threshold, and its successor g that meets the function correlation threshold.
[0022] Further, the using unit testing in the extension unit to discover potential errors includes:
[0023] For each group of extension units related to the target function in the program under test P, the test driver driver_e and symbol stub stub_g of the extension unit are automatically generated;
[0024] The test driver driver_e assigns symbolic values to the input variables of e according to the input type of the calling function e, and then calls function e; when the successor function g is executed, it is replaced by the symbolic stub stub_g to make the return value have symbolic meaning;
[0025] Concolic unit testing is performed on the extension unit to explore crash errors related to the target function f. During this execution, a set of inputs is generated for each execution path of the test driver driver_e, and the execution path and the constraints of the path are recorded when a crash error is encountered.
[0026] Furthermore, the target-driven search strategy is used to verify whether the potential error exists globally, including:
[0027] Analyze the entire program to construct a control flow graph CFG and a function call graph CG, use potential errors as targets to mark them in the generated CFG and CG to obtain a control flow graph containing potential target marked nodes;
[0028] In the system-level concolic testing phase, selected inputs and target-driven search strategies are used to quickly locate whether local errors exist globally, and to obtain inputs of global paths and related crash reports.
[0029] Furthermore, the target-driven search strategy includes:
[0030] When the execution path runs to a potential target address, the solver is used to determine whether the local input of the potential error satisfies the system-level path constraints; if so, continue to run this path until a global crash is triggered and a global test case is output; if the execution path cannot find a solution that satisfies the path constraints among all potential errors, the path is discarded and continues to look for the next system-level input that can trigger the unit test entry.
[0031] A target-driven software testing device using the above method comprises:
[0032] The extension unit construction module is used to collect the calling relationships between functions, calculate the correlation of functions by using the calling relationships between functions, and construct the extension unit by the correlation of functions;
[0033] Unit testing module, used to find potential errors in extension units using unit testing;
[0034] A system-level error verification module is used to verify whether potential errors exist globally using a goal-driven search strategy.
[0035] The advantages and beneficial effects of the present invention are as follows:
[0036] Since the unit test of a single objective function often lacks the context related to variables, most of the local errors found by this unit test are false alarms. To find the real errors in these potential error warnings containing a large number of false alarms, it is necessary for the on-site test engineers to complete a large amount of manual testing and other work, which is extremely inefficient. However, the present invention expands the single objective function into a group of test units by calculating the function dependency relationship. Since this group of test units contains the function information that is relatively close to its dependency relationship, the false alarms caused by the lack of context are greatly reduced. On the other hand, since the concolic execution of the system level needs to be searched globally in the program under test, and the programs in the real world are relatively complex and face the problem of path explosion, it is difficult to find the errors existing in the objective function under such conditions. However, the present invention reduces the difficulty of testing the objective function and improves the accuracy of the test by first performing error search locally and then putting the potential error information into the concolic execution of the system level for verification. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 This is the overall block diagram of TDCT.
[0038] Figure 2 This is a schematic diagram of the expansion unit of the example function f7. DETAILED DESCRIPTION
[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific embodiments and drawings.
[0040] In order to make full use of the advantages of unit testing and concolic testing, the present invention develops an automatic testing system TDCT (target-driven concolic testing). TDCT builds extension units by using the correlation of functions, and uses unit testing to quickly identify potential errors in the extension units. In addition, TDCT also focuses on verifying these potential errors and generating system-level test inputs in system-level concolic testing. TDCT works by performing system-level concolic testing in a limited time to collect the calling relationships between functions and build extension units through correlation with target functions. Unit testing is then used in the extension units to discover potential errors. Finally, a target-driven search strategy is used in system-level concolic testing to quickly verify whether local potential errors exist globally.
[0041] 1. Overall process
[0042] The overall framework of TDCT is shown in Figure 1 , including three modules: building extension units, unit testing using extension units, and system-level error verification.
[0043] (1) Constructing the extension unit: TDCT starts with the target program to be tested and a set of system-level test initial values as input, and sets a time threshold (such as 1 hour, etc.) according to the actual situation. In this stage, the concolic engine aims to obtain as many system-level paths as possible. Once the set time threshold is reached, it stops running. In this process, it is also necessary to collect the call relationship of the functions on the running path (that is, the call frequency between functions). TDCT uses the obtained function call frequency to calculate the function correlation, and presets the function correlation threshold according to the actual situation, so as to select the relevant functions that meet the requirements to construct the test unit.
[0044] (2) Unit testing using extended units: TDCT generates test drivers and symbolic stub functions for the test unit built in step (1), and runs concolic unit tests on it to explore suspected errors in the unit and report potential errors in the unit. The test unit in this step extends the test target by utilizing the correlation of function calls, which can effectively reduce the false positive rate of unit testing.
[0045] (3) System-level error verification: Although step (2) has significantly reduced the false positives of unit tests, false positives still exist due to the lack of global context information. Therefore, TDCT ultimately adopts system-level concolic testing, targeting the potential errors reported in step (2) and implementing target-based concolic testing to further confirm the real errors, thereby reducing the large amount of manual operations that testers need to complete.
[0046] 2. Specific implementation steps
[0047] Step 1: Build the Extension Unit
[0048] (1) Calculate function correlation
[0049] TDCT starts concolic execution with the target program to be tested and a set of system-level test initial values as input, and sets a time threshold (such as 1 hour) according to the actual situation. The purpose of this stage is to obtain as many system-level paths as possible. Once the set time threshold is reached, it stops running. In this process, it is also necessary to collect the calling relationship of the functions on the running path (that is, the calling frequency between functions), so as to use the observed function calling frequency to determine the dependency between functions by calculating the conditional probability. Generally, if two functions call each other frequently, they are considered to have a high degree of mutual dependency. If TDCT often observes that function f1 calls f2 or f2 calls f1 here, TDCT marks f1 and f2 as highly correlated.
[0050] It should be noted that the longer the running time (the longer the time threshold is set), the more accurate the function correlation obtained may be. However, it is found through experiments that the initial stage of concolic testing contributes the most to the increase of code coverage. After a period of time, the rate of increase of code coverage will decrease sharply. Therefore, it is necessary for test engineers to strike a balance between time cost and accuracy. In addition, the initial input is usually provided by the developer. Including the input that can reach the target function in the initial input set as much as possible can more accurately reflect the correlation between the target function and its nearby functions.
[0051] TDCT runs concolic execution to obtain as many system-level paths as possible and calculates the calling frequency between functions. For example, assuming that the goal is to complete the test of f7 in the program under test P and obtain a global test case for the error of this function, in order to minimize false positives and manual operations of on-site testers, TDCT performs concolic execution on the program under test P for a certain period of time and records the calling frequency between functions during the process, such as Figure 2 As shown, the following test paths exist after concolic execution ends:
[0052] P1:main→f1→f3→f6→f8→f 10
[0053] P2:main→f1→f4→f7→f8→f 10
[0054] P3:main→f1→f4→f7→f9→f 11
[0055] P4:main→f2→f7→f8→f 10
[0056] P5:main→f2→f7→f9→f 11
[0057] P6:main→f2→f5→f7→f8→f 10
[0058] P7:main→f2→f5→f7→f9→f 11
[0059] P8:main→f2→f5→f9→f 11
[0060] Therefore, if you want to calculate the calling frequency between f7 and f9 functions, the calculation method is to calculate the total number of times f7 and f9 appear in all paths, that is, F(f7)=6, F(f9)=4. At the same time, calculate the number of calls between f7 and f9, that is, the sum of the number of times f7 calls f9 and f9 calls f7, that is, F(f7,f9)=3.
[0061] After obtaining the calling frequency between functions, the correlation R between two functions x and y can be calculated by the following formula:
[0062]
[0063] Among them, F(x,y) represents the sum of the number of times function x calls function y and the number of times function y calls function x, F(x) represents the number of times function x appears in all paths, and F(y) represents the number of times function y appears in all paths.
[0064] In the example above, It is worth noting that the path selection algorithm and execution time used in the system-level concolic execution will significantly affect the calculation of dependencies. Generally, the more paths are executed near the target function and the longer the running time, the higher the accuracy of the function dependencies.
[0065] (2) Construction of extension unit
[0066] Due to the lack of context information, unit testing usually brings a lot of false positives. In order to filter out some false positives, TDCT expands the test object from a single function to a set of test units obtained by calculating function correlation. After having the correlation between functions, TDCT will set the correlation threshold of the function according to the actual situation, and then construct the expansion unit related to the tested target based on this value. For each target function f, TDCT constructs a set of expansion units, which includes f itself, the caller e that meets the correlation threshold, and its successor g that meets the correlation threshold.
[0067] For the successor g, function f is highly dependent on all function nodes on the path from f to g. That is, the dependencies between all functions that directly call each other on the path from f to g exceed the threshold. For example, suppose the target function is f7 and the dependency threshold is set to 0.7. Figure 2 The correlation of R(f7, f8) is 0.8, R(f7, f9) is 0.9, and R(f8, f 10 ) is 0.9, R(f9,f 11 ) is 0.3. Therefore, in addition to the f7 function itself, the functions f8, f9 and f 10 will be added to the extension unit of f7.
[0068] For a caller e, TDCT starts from function f itself and searches for caller functions upward until it encounters a function e whose caller correlation with function e is lower than a preset correlation threshold. The number of function e will determine the number of extended units for the next stage of unit testing. For example, Figure 2 It indicates that there are two callers e (i.e., functions f1 and f2) of the target f (function f7), so two groups of extension units {f1, f4, f7, f8, f 10 , f9} and {f2, f7, f8, f 10 , f9}.
[0069] It is worth noting that the relevance threshold is closely related to the scope of the expansion unit. When the threshold is high, the expansion unit will become smaller until it becomes a single target function in the extreme case; when the threshold is low, more functions will be added to the expansion unit until it becomes the entire program in the extreme case. In other words, the relevance threshold affects the false positive rate of unit testing on the one hand, and on the other hand, it also affects performance and even leads to a large number of false positives (due to the increase in the number of functions, the complexity of the expansion unit increases, which makes it difficult to fully cover all paths in unit testing).
[0070] Step 2: Unit testing using the extension unit
[0071] Concolic testing is a program analysis technique that can systematically explore paths in a program. TDCT uses concolic testing to comprehensively explore extension units. For each set of extension units related to the target function in the program under test P, TDCT automatically generates a test driver driver_e and a symbolic stub stub_g for the extension unit.
[0072] After constructing the executed unit, TDCT generates a symbolic unit test driver driver_e for the entry function e to start the unit test, and generates a symbolic stub stub_g for the successor function g. The test driver assigns symbolic values to the input variables of e according to the input type of the calling function e, and then calls function e. When function g is executed, it is replaced by the symbolic stub stub_g to make the return value have symbolic meaning. For simplicity, the symbolic stub only returns symbolic values without updating global variables and outputs. The test driver and symbolic stub usually over-approximate the actual environment of function f and cause infeasible expansion unit paths, causing false positives, but compared with unit testing the target function f alone, the false positives of TDCT have been significantly reduced.
[0073] Table 1 shows the test driver driver_e and symbol stub stub_g of an extension unit automatically generated by TDCT. Function driver_e uses the symbol variable str generated in function make_symbolic to call function e, and function stub_g uses the symbol variable str generated in function make_symbolic to return a symbolic value instead of a specific value. All places where function g is called need to be replaced with stub_g to make its return value symbolic.
[0074] Table 1. Example of test driver and symbol stubs
[0075]
[0076] Unit testing often results in a large number of false positives due to the lack of contextual information about variables. To filter out most false positives, TDCT computes the correlation between functions by running the program over a period of time and expands the test object to an extension unit consisting of multiple functions. After constructing the extension unit, it generates a test driver driver_e for the entry function e and a symbolic stub stub_g for the successor function g, and starts unit testing the constructed extension unit. TDCT performs concolic unit testing on the extension unit to explore crash errors associated with function f. During this execution, TDCT generates a set of inputs for each execution path of the test driver driver_e and records the execution path and the constraints of the path when a crash error is encountered.
[0077] On the one hand, compared with the unit test of a single target function, TDCT has a lower false positive rate due to the addition of functions that have a strong dependency on the target function, that is, the predecessor constraints of the related functions are added to the extended unit. On the other hand, compared with system-level concolic testing (i.e., special unit testing that takes the entire program as a unit), TDCT reduces the search space while ensuring accuracy and focuses on the testing of the code near the target function.
[0078] Step 3: System-level error verification
[0079] In step 2, TDCT obtains information related to the local crash errors of the target function in the program under test, such as the local inputs and path constraints where the errors occur. TDCT uses the crash-related inputs and path constraints to explore whether there are system-level inputs that may cause local crashes, in order to verify whether the potential crash found in step 2 is real.
[0080] (1) State fork
[0081] Use a tool such as IDA Pro to statically analyze the entire program to build a control flow graph (CFG) and function call graph (CG). Use the potential errors in step 2 as targets and mark them in the generated CFG and CG to obtain a control flow graph containing potential target marked nodes.
[0082] (2) System-level concolic testing
[0083] In the system-level concolic testing phase, TDCT uses selected inputs and a target-based search strategy to quickly locate whether local errors exist globally and obtain inputs of global paths and related crash reports.
[0084] During the execution of a path, a new state is generated at the location of the conditional branch. TDCT maps these new states to the next block address and puts them into the candidate pool (or candidate list). After the current state is executed, TDCT will select the appropriate path from the candidate pool according to the target-based path selection algorithm. In particular, when the execution path runs to a potential target address, that is, when the path runs to the unit test entry e, TDCT will use the solver to determine whether the local input of the potential error satisfies the system-level path constraint. If it can be satisfied, it will continue to run this path until a global crash is triggered and a global test case is output; if the execution path cannot find a solution that satisfies the path constraint among all potential errors, it will discard the path and continue to look for the next system-level input that can trigger the unit test entry.
[0085] The target-driven search strategy of the present invention can be implemented by the following steps:
[0086] When a path is executed, determine whether the next instruction address of the new state in the candidate list hits the target function, that is, the function where the patch is located; if it hits, assign the highest priority to all the new states that hit, and select a path closest to the core target in the new state with the highest priority to continue running. After the path is executed, the execution path will continue to be selected in the new state with the highest priority; if there is no hit, or the new state with the highest priority is fully executed, the upper-level calling function of the target function is found in combination with the control flow graph CFG and the function call graph CG, and confirm whether there is a new state in the candidate list that hits the upper-level calling function; select a new state closest to the call point in the new state that hits the upper-level calling function to continue execution. Note: Except for the new state that hits the target function, other new states do not have priority. Since it has been confirmed whether the previously generated new state hits the target function or its caller, the new states derived from the current running state are given priority to confirm whether they hit the target function.
[0087] You can use formula D i =|state i _addr-target_addr|Calculate the distance between the new state and the target, and select the new state closest to the target. Where Di represents the distance, state i _addr indicates the next instruction address of the new state, and target_addr indicates the target address. For target_addr, it is first the address of the patch line determined before running the program, and when no new state hits the target function, it is the address of the function call line in the call chain of the target function.
[0088] After that, continue to perform all the above steps for the next core target code. The purpose of running all new states that hit the target function (i.e., running all new states with the highest priority) is to test as much code as possible near the patch. Because for the security testing of the patch program, it is usually not enough to focus only on the paths covered by the patch code itself, and the code near the patch is usually associated with the patch, so it is necessary to perform a more comprehensive test on the function where the patch is located.
[0089] The TDCT of the present invention uses the functional dependencies in the tested software to determine the scope of the unit test, and utilizes concolic execution to perform unit testing on the tested unit to quickly identify local errors. Unit testing combined with concolic execution can quickly identify suspicious errors in the tested unit, but due to the lack of global context information, it will inevitably bring a large number of false positives. In addition, TDCT also focuses on verifying these potential errors and generating system-level test inputs in system-level concolic testing. In this way, TDCT does not explore a large amount of symbolic search space, but focuses on extending local errors to system-level paths, thereby accurately finding errors in system-level concolic testing and outputting system-level test cases. TDCT has a high system-level error detection capability and a high detection accuracy.
[0090] The key points of the present invention mainly include:
[0091] The first key point is to propose a method to calculate function call correlation, which is the content in step 1.
[0092] The second key point is to propose a method to construct test units using function dependencies, which is the content in step 1.
[0093] The third key point is to propose a method to unit test the extension unit using concolic testing, which is the content in step 2.
[0094] The fourth key point is to propose a method to verify the potential errors in step 2 by using goal-driven concolic testing, which is the content of step 3.
[0095] In the step 1 of the above method, the correlation part of the function is calculated. The present invention adopts a system-level concolic test method to obtain the function call frequency. Other effective test methods can also be used to obtain the data.
[0096] In the step 1 of the above method, the function correlation part is calculated. The present invention uses conditional probability to calculate the dependency relationship. Other effective calculation formulas can also be used to obtain the dependency relationship.
[0097] In step 2 of the above method, the local potential error part is obtained. The present invention uses unit testing based on concolic testing to find potential errors. Other effective testing methods can also be used to obtain it.
[0098] Based on the same inventive concept, another embodiment of the present invention provides a target-driven software testing device using the above method, which includes:
[0099] The extension unit construction module is used to collect the calling relationships between functions, calculate the correlation of functions by using the calling relationships between functions, and construct the extension unit by the correlation of functions;
[0100] Unit testing module, used to find potential errors in extension units using unit testing;
[0101] A system-level error verification module is used to verify whether potential errors exist globally using a goal-driven search strategy.
[0102] Based on the same inventive concept, another embodiment of the present invention provides an electronic device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.
[0103] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, CD), which stores a computer program. When the computer program is executed by a computer, it implements the various steps of the method of the present invention.
[0104] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.
Claims
1. A software testing method based on goal-driven, characterized in that: The following steps are involved: Collect the calling relationships between functions, use the calling relationships between functions to calculate the correlation of functions, and build an expansion unit based on the correlation of functions; Use unit tests to find potential bugs in your extension units; Use a goal-driven search strategy to verify whether potential errors exist globally; The collecting the calling relationships between functions and calculating the correlation of the functions by using the calling relationships between the functions includes: The concolic execution starts with the target program to be tested and a set of system-level test initial values as input, and the time threshold is set according to the actual situation. The concolic engine aims to obtain as many system-level paths as possible and stops running once the set time threshold is reached. At the same time, the calling relationship of the functions on the running path is collected, and the calling frequency between functions is used to determine the correlation between functions by calculating the conditional probability. The use of unit tests in extension units to discover potential errors includes: For each group of extension units related to the target function in the program under test P, the test driver driver_e and symbol stub stub_g of the extension unit are automatically generated; The test driver driver_e assigns symbolic values to the input variables of e according to the input type of the calling function e, and then calls function e; when the successor function g is executed, it is replaced by the symbolic stub stub_g to make the return value have symbolic meaning; Concolic unit testing is performed on the extension unit to explore crash errors related to the target function f. During this period, a set of inputs is generated for each execution path of the test driver driver_e, and the execution path and the constraints of the path are recorded when a crash error is encountered. The target-driven search strategy is used to verify whether potential errors exist globally, including: Analyze the entire program to construct a control flow graph CFG and a function call graph CG, use potential errors as targets to mark them in the generated CFG and CG to obtain a control flow graph containing potential target marked nodes; In the system-level concolic testing phase, the selected input and target-driven search strategies are used to quickly locate whether local errors exist globally, and obtain the input of the global path and related crash reports; The goal-driven search strategy includes: When a path is executed, determine whether the next instruction address of the new state in the candidate list hits the target function, that is, the function where the patch is located; if it hits, assign the highest priority to all the new states that hit, and select a path closest to the core target in the new state with the highest priority to continue running. After executing the path, continue to select the execution path in the new state with the highest priority; if there is no hit, or the new state with the highest priority is all executed, then combine the control flow graph CFG and the function call graph CG to find the upper-level calling function of the target function, and confirm whether there is a new state in the candidate list that hits the upper-level calling function; if so, select a new state closest to the call point in the new state that hits the upper-level calling function to continue execution.
2. The method according to claim 1, characterized in that After obtaining the calling frequencies between the functions, the correlation R(x, y) between the two functions x and y is calculated by the following formula: Among them, F(x,y) represents the sum of the number of times function x calls function y and the number of times function y calls function x, F(x) represents the number of times function x appears in all paths, and F(y) represents the number of times function y appears in all paths.
3. The method according to claim 1, characterized in that: The expansion unit is constructed by the correlation of functions, including: A function correlation threshold is preset according to actual conditions, and related functions that meet the requirements are selected according to the function correlation threshold to construct an extension unit; for each target function f, a set of extension units is constructed, including the target function f itself, the caller e that meets the function correlation threshold, and its successor g that meets the function correlation threshold.
4. A target-driven software testing device using the method according to any one of claims 1 to 3, characterized in that: include: The extension unit construction module is used to collect the calling relationships between functions, calculate the correlation of functions by using the calling relationships between functions, and construct the extension unit by the correlation of functions; Unit testing module, used to find potential errors in extension units using unit testing; A system-level error verification module is used to verify whether potential errors exist globally using a goal-driven search strategy.
5. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 3 is implemented.
