A unit test case generation method, device and medium
The source code is parsed and analyzed through the LLVM compiler and generated test cases, solving the problem of inefficient writing unit tests manually, and achieving automated and efficient test case generation, reducing development costs and delivery time.
Patent Information
- Application Number
- CN202411186592.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-28
AI Technical Summary
Traditional manual writing of unit tests is inefficient, especially when faced with large, complex software systems, developers need to spend a lot of time and effort creating and maintaining test cases, increasing development costs and potentially delaying software delivery time.
The source code is parsed through the compiler LLVM, and the intermediate representation is generated, code analysis and boundary analysis are performed. The execution path of the program is automatically explored using symbolic execution technology, and the test cases are generated, and iteratively optimized based on the test coverage, and finally the required tests are generated. Use cases.
It improves the degree of automation and efficiency of tests, enhances the comprehensiveness and effectiveness of tests, reduces the need to manually construct test cases, promptly detects and fixes potential problems, and reduces development and maintenance costs.
Smart Images

Figure CN118689780B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer software technology, and in particular to a method, device and medium for generating unit test cases. Background Art
[0002] Today, information technology is changing with each passing day. The scale and complexity of software systems are growing at an unprecedented rate. Software has become a key force in promoting social progress and industrial upgrading. However, this development trend has also put forward more stringent requirements on the quality and reliability of software. As an important link in ensuring software quality, software testing has become increasingly prominent.
[0003] Unit testing focuses on the smallest testable unit in the software, such as a function, method, or module, and aims to verify that these independent units behave as expected in isolation.
[0004] However, the traditional method of manually writing unit tests faces challenges and shortcomings. Manually writing unit tests is inefficient, especially when facing large and complex software systems. Developers need to spend a lot of time and energy to create and maintain test cases, which not only increases development costs but may also delay software delivery time. Summary of the invention
[0005] The embodiments of the present application provide a unit test case generation method, device and medium for solving the following technical problems: Manually writing unit tests is inefficient, especially when facing large and complex software systems. Developers need to spend a lot of time and energy to create and maintain test cases, which not only increases development costs but also may delay software delivery time.
[0006] The present application embodiment adopts the following technical solutions:
[0007] The embodiment of the present application provides a method for generating unit test cases. The method comprises: parsing the acquired source code through the compiler LLVM to generate an intermediate representation; performing code analysis and boundary analysis on the intermediate representation to obtain the unit to be tested; replacing the input variable with a symbolic value, and collecting path constraints for different paths corresponding to the unit to be tested through the symbolic value; obtaining test case information based on the collected path constraints, determining a corresponding test framework based on the test case information, and generating a reference test case; executing the reference test case, and determining that the reference test case is a required test case when the test coverage meets a preset coverage threshold.
[0008] The intermediate identifier in the embodiment of the present application is a machine- and language-independent intermediate code representation form, which can perform code analysis and conversion across different platforms and architectures, thereby improving the portability and reusability of the code. By performing code analysis on the intermediate representation, the structure and behavior of the program can be understood more accurately, ensuring that the test case can cover the critical path and boundary conditions of the code. Through symbolic execution technology, the execution path of the program is automatically explored without manually constructing a large number of test cases. By executing test cases, the correctness and stability of the program are verified, and potential problems are discovered and fixed in a timely manner. The embodiment of the present application parses and analyzes the source code through the compiler LLVM, and generates test cases, which not only improves the automation and efficiency of the test, but also enhances the comprehensiveness and effectiveness of the test.
[0009] In one implementation of the present application, code analysis and boundary analysis are performed on the intermediate representation to obtain a unit to be tested, specifically including: traversing the function definition nodes in the intermediate representation, and recording the function definition information corresponding to each function definition node; wherein the function definition information includes at least one of the name, parameters, and return type; constructing a control flow graph for the intermediate representation to perform control flow analysis through the control flow graph to obtain data flow information; determining code data with boundaries in the intermediate representation, and determining boundary conditions; wherein the code data with boundaries includes at least one of loop statements, conditional statements, array accesses, and pointer dereferences; based on the function definition information, data flow information, and boundary conditions, obtaining a unit to be tested; wherein the unit to be tested includes at least one of the key function, the boundary condition, and the exception path.
[0010] In one implementation of the present application, input variables are replaced with symbolic values, and path constraints are collected for different paths corresponding to the unit to be tested through the symbolic values, specifically including: determining the control flow graph corresponding to the unit to be tested; replacing the input variables with symbolic values, so that the symbolic values traverse the control flow graph, so as to generate different execution paths based on each conditional branch in the control flow graph; based on the branch points of the execution path, generating corresponding conditional expressions, and using the conditional expressions as branch point constraints; based on the multiple branch point constraints corresponding to the unit to be tested, obtaining the path constraints corresponding to the test unit.
[0011] In one implementation of the present application, test case information is obtained based on the collected path constraints, and a corresponding test framework is determined based on the test case information to generate a reference test case, which specifically includes: inputting the path constraints corresponding to the unit to be tested into the constraint solver to obtain input values that trigger different execution paths; obtaining test case information based on the input values; matching the test case information with a preset test framework reference table to determine a reference test framework; wherein the preset test framework reference table includes multiple test frameworks, and also includes integration conditions corresponding to the multiple test frameworks; the integration conditions include at least one of integration capability, language type required for integration, extensibility, and maintainability; in the reference test framework, a reference test case is generated according to the test case information.
[0012] In one implementation of the present application, a reference test case is executed, and when the test coverage meets a preset coverage threshold, the reference test case is determined to be the required test case, specifically including: executing the reference test case to obtain the test coverage; comparing the test coverage with the preset coverage threshold, and when the test coverage is less than the preset coverage threshold, determining the paths that are not covered by the test; generating new test cases for the paths that are not covered by the test, and generating new reference test cases based on the new test cases and the reference test cases to perform test case generation iterations; executing the new reference test case until the test coverage corresponding to the new reference test case is greater than the preset coverage threshold, ending the iterative process, and obtaining the required test case.
[0013] In one implementation of the present application, after determining a corresponding test framework based on test case information and generating a reference test case, the method also includes: vectorizing an execution path corresponding to the reference test case; calculating the similarity between the vectorized execution path and a preset defect data vector to determine reference defect data that matches the reference test case based on the similarity result; dividing the reference test case into multiple test case groups based on the matching relationship; based on each test case group, taking each test case path as a node, simulating the execution of the test case path for the test case, and assigning weights to each node based on the simulation results; and screening the reference test cases based on the weight assignment results and the weights corresponding to each reference defect data.
[0014] In one implementation of the present application, the execution path corresponding to the reference test case is vectorized, specifically including: constructing a reference test case set based on each reference test case; determining the abnormal score corresponding to each reference test case based on the isolation forest algorithm, and screening out the abnormal reference test cases in the reference test case set based on the abnormal score; performing feature extraction on the path of each reference test case after screening to obtain the data feature value corresponding to each reference test case; combining each data feature value to obtain a data feature set corresponding to the reference test case set; projecting each data feature value in the data feature set to obtain a first data feature vector corresponding to each data feature value; and assigning weights to different data features in the first data feature vector based on the difference in data features to obtain a second data feature vector corresponding to each test case.
[0015] In one implementation of the present application, the obtained source code is parsed by the compiler LLVM to generate an intermediate representation, which specifically includes: preprocessing the obtained source code by the compiler LLVM; wherein the preprocessing includes macro definition replacement and header file import; cutting the preprocessed source code, and combining word sequences based on the cut source code to construct a syntax tree; and traversing the syntax tree from top to bottom by a code generator to generate an intermediate representation.
[0016] An embodiment of the present application provides a unit test case generation device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: parse the acquired source code through the compiler LLVM to generate an intermediate representation; perform code analysis and boundary analysis on the intermediate representation to obtain a unit to be tested; replace input variables with symbolic values, and collect path constraints for different paths corresponding to the unit to be tested through the symbolic values; obtain test case information based on the collected path constraints, determine a corresponding test framework based on the test case information, and generate a reference test case; execute the reference test case, and determine that the reference test case is a required test case when the test coverage meets a preset coverage threshold.
[0017] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, wherein the computer executable instructions are configured to: parse an acquired source code through a compiler LLVM to generate an intermediate representation; perform code analysis and boundary analysis on the intermediate representation to obtain a unit to be tested; replace input variables with symbolic values, and collect path constraints for different paths corresponding to the unit to be tested through the symbolic values; obtain test case information based on the collected path constraints, determine a corresponding test framework based on the test case information, and generate a reference test case; execute the reference test case, and determine that the reference test case is a required test case when the test coverage meets a preset coverage threshold.
[0018] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the intermediate identifier in the embodiments of the present application is a machine- and language-independent intermediate code representation form, which can perform code analysis and conversion across different platforms and architectures, thereby improving the portability and reusability of the code. By performing code analysis on the intermediate representation, the structure and behavior of the program can be understood more accurately, ensuring that the test cases can cover the critical paths and boundary conditions of the code. Through symbolic execution technology, the execution path of the program is automatically explored without manually constructing a large number of test cases. By executing test cases, the correctness and stability of the program are verified, and potential problems are discovered and fixed in a timely manner. The embodiments of the present application parse and analyze the source code through the compiler LLVM, and generate test cases, which not only improves the automation and efficiency of the test, but also enhances the comprehensiveness and effectiveness of the test. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:
[0020] Figure 1 A flow chart of a method for generating unit test cases provided in an embodiment of the present application;
[0021] Figure 2 A unit test case generation example diagram provided in an embodiment of the present application;
[0022] Figure 3 A schematic diagram of the structure of a unit test case generation device provided in an embodiment of the present application.
[0023] Reference numerals:
[0024] 200: unit test case generating device, 201: processor, 202: memory. DETAILED DESCRIPTION
[0025] The embodiments of the present application provide a method, device and medium for generating a unit test case.
[0026] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.
[0027] The technical solution proposed in the embodiment of the present invention is described in detail below with reference to the accompanying drawings.
[0028] Figure 1 A flow chart of a method for generating unit test cases provided in an embodiment of the present application is as follows: Figure 1 As shown, the unit test case generation method includes the following steps:
[0029] Step 101: parse the acquired source code through the compiler LLVM to generate an intermediate representation.
[0030] In one embodiment of the present application, the obtained source code is preprocessed by the compiler LLVM; wherein the preprocessing includes macro definition replacement and header file import. The preprocessed source code is segmented, and word sequences are combined based on the segmented source code to construct a syntax tree. The syntax tree is traversed from top to bottom by a code generator to generate an intermediate representation.
[0031] Specifically, the compiler LLVM (Low Level Virtual Machine) is an open source compiler infrastructure and toolkit that provides a rich set of tools and libraries for compiling, optimizing, and generating code for different programming languages and target platforms. LLVM is designed to provide a high degree of portability, scalability, and performance optimization capabilities.
[0032] Specifically, the source code processing flow of LLVM in the embodiment of the present application can be divided into the following stages:
[0033] Preprocessing stage: This stage includes macro replacement, header file import, etc. After preprocessing, the macro definitions in the code will be replaced and the contents of the header file will be included.
[0034] Compilation phase: The compilation phase includes lexical analysis and syntax analysis. Lexical analysis is to cut the code into multiple tokens, such as brackets, equal signs, strings, etc. Syntax analysis is to combine word sequences into various grammatical phrases (such as programs, statements, expressions, etc.) based on lexical analysis, and build an abstract syntax tree. The syntax analysis program will determine whether the source program is structurally correct.
[0035] Generate intermediate code IR: The code generator traverses the syntax tree from top to bottom and gradually translates it into LLVM IR (intermediate representation). LLVM IR is a low-level, stack-based intermediate representation that is suitable for various programming languages and target platforms.
[0036] IR optimization: LLVM's optimizer optimizes the intermediate representation to improve performance and code quality. The optimization process includes constant propagation, dead code elimination, loop unrolling, etc.
[0037] Generate target code: The optimized intermediate representation is further converted into assembly code or machine code of the target platform.
[0038] Linking: LLVM will link the compiled .o files (object files) with the required dynamic and static libraries to generate an executable file.
[0039] When using the LLVM compiler to compile source code into an executable file, the source code first undergoes lexical analysis and syntax analysis to generate an abstract syntax tree, which ensures the correctness of the code structure. Subsequently, the syntax tree is further converted into LLVM's intermediate representation, which not only retains the semantic information of the source code, but also represents the program in a platform-independent and easy-to-optimize form. In the LLVM IR phase, the compiler analyzes the structure of the code in detail, including the definition of the function, the type and number of parameters, the statement logic within the function body, and the return value type, and executes various optimization strategies based on this information to improve the execution efficiency of the code. Since LLVM provides deep insights into the code structure, parameters, statements, and return values during the compilation process, it becomes an ideal platform for automatically adding unit tests at compile time or after compilation.
[0040] Step 102: Perform code analysis and boundary analysis on the intermediate representation to obtain a unit to be tested.
[0041] In one embodiment of the present application, the function definition nodes in the intermediate representation are traversed, and the function definition information corresponding to each function definition node is recorded; wherein the function definition information includes at least one of the name, parameters, and return type. A control flow graph is constructed for the intermediate representation to perform control flow analysis through the control flow graph to obtain data flow information. Code data with boundaries is determined in the intermediate representation, and boundary conditions are determined; wherein the code data with boundaries includes at least one of loop statements, conditional statements, array access, and pointer dereference. Based on the function definition information, data flow information, and boundary conditions, a unit to be tested is obtained; wherein the unit to be tested includes at least one of the key function, boundary conditions, and abnormal path.
[0042] Specifically, the intermediate representation is analyzed at the function level, and the structure of the code is analyzed in detail, including the definition, parameters, control flow, and data flow of the function. Based on the code analysis, the key functions, boundary conditions, and abnormal paths that need to be tested are identified.
[0043] Further, all function definition nodes in the intermediate representation are traversed, and the name, parameter list and return type of each function are recorded. A control flow graph is constructed, which is a graphical representation that shows all possible control flow paths in a program. The nodes of a control flow graph usually correspond to statements or instruction blocks in a program, and the edges represent possible transfers of control flow. Secondly, in the embodiment of the present application, boundary conditions refer to specific input values or conditions that may cause abnormal or erroneous program behavior, such as the boundaries of loops, the boundaries of conditional statements, the boundaries of array accesses, etc. The identified key functions (such as the core part of the business logic), the code paths involving boundary conditions, and the paths that may cause exceptions are used as units to be tested.
[0044] Step 103: Replace the input variable with a symbolic value, and collect path constraint conditions for different paths corresponding to the unit to be tested respectively through the symbolic value.
[0045] In one embodiment of the present application, a control flow graph corresponding to the unit to be tested is determined. Input variables are replaced with symbolic values, and the symbolic values are traversed through the control flow graph to generate different execution paths based on the conditional branches in the control flow graph. Based on the branch points of the execution path, corresponding conditional expressions are generated, and the conditional expressions are used as branch point constraints. Based on the multiple branch point constraints corresponding to the unit to be tested, the path constraints corresponding to the test unit are obtained.
[0046] Specifically, the embodiment of the present application utilizes symbolic execution technology to replace input variables with symbolic values and track the propagation and changes of symbolic values in the program. Through symbolic execution, different execution paths of the program are explored and path constraints are collected.
[0047] Furthermore, the embodiment of the present application constructs a control flow graph for the unit to be tested through static analysis or compiler technology. In symbolic execution, the input variables are replaced with symbolic values, which represent arbitrary values in the input space. By simulating the execution of the program and using symbolic values instead of actual inputs, all paths in the control flow graph are traversed. During the traversal process, different execution paths are generated according to conditional branches, and changes in symbolic values are tracked. At each branch point, that is, a conditional statement, a conditional expression is generated, which describes the conditions that cause the program to execute along a specific path. These conditional expressions are used as constraints on branch points to describe the execution conditions of different paths.
[0048] Furthermore, for each possible execution path, the constraints of all branch points are collected and combined into complete path constraints of the path. Based on the path constraints, specific test cases are generated, which will cover all possible execution paths of the program.
[0049] Step 104: Based on the collected path constraints, test case information is obtained, a corresponding test framework is determined based on the test case information, and a reference test case is generated.
[0050] In one embodiment of the present application, the path constraint conditions corresponding to the unit to be tested are input into the constraint solver to obtain input values that trigger different execution paths. Test case information is obtained based on the input values. The test case information is matched with a preset test framework reference table to determine a reference test framework; wherein the preset test framework reference table includes a variety of test frameworks, and also includes integration conditions corresponding to the various test frameworks; the integration conditions include at least one of integration capability, language type required for integration, extensibility, and maintainability. In the reference test framework, a reference test case is generated according to the test case information.
[0051] Specifically, use the constraint solver to solve the path constraints, generate test cases that meet the conditions, select the test framework to be used based on the generated test cases, and write or automatically generate test code, including preparation of test data, calling of test functions, assertions, etc.
[0052] Furthermore, in the symbolic execution process of the embodiment of the present application, a set of path constraints are generated for each possible execution path, and the generated path constraints describe all necessary conditions for the program to execute along the path. The path constraints are taken as input, input into the constraint solver, and the actual input values that meet these conditions are found. The actual input values returned by the constraint solver can be directly used to generate test cases, which include input values, expected outputs, execution paths, etc., to guide test execution and result verification.
[0053] Furthermore, the embodiment of the present application is provided with a preset test framework reference table, which contains a table of multiple test frameworks and their integration conditions. The integration conditions include factors such as the framework's integration capabilities, supported language types, extensibility, and maintainability. According to the test case information, such as the test type, required test language, expected extensibility, etc., the most suitable test framework is found in the reference table.
[0054] For example, the pre-built test framework reference table includes test frameworks such as JUnit (Java), pytest (Python), and NUnit (.NET). If the unit to be tested is written in Python and a test framework that supports high customization and extensibility is required, pytest is the best choice.
[0055] Furthermore, based on the above matching results, a suitable test framework is selected, and in the selected test framework, corresponding test case codes are written or generated according to the test case information.
[0056] In one embodiment of the present application, the execution path corresponding to the reference test case is vectorized. The execution path after vectorization is calculated for similarity with the preset defect data vector to determine the reference defect data matching the reference test case based on the similarity result. Based on the matching relationship, the reference test case is divided into multiple test case groups. Based on each test case group, each test case path is taken as a node, and the test case is simulated to execute the test case path, and based on the simulation result, weights are assigned to each node. Based on the weight assignment result and the weight corresponding to each reference defect data, the reference test case is screened.
[0057] Specifically, each test case corresponds to one or more execution paths, which describe the code sequences that the program may go through when executing the test case, and map each element in the path (such as function calls, conditional branches, etc.) to points or features in the vector space, thereby converting the execution path into a vector form. The embodiment of the present application is provided with a preset defect data vector, which is a description of a set of known defects and has also been vectorized. The similarity between the execution path vector and the preset defect data vector is calculated by cosine similarity.
[0058] For example, assuming there is a preset defect data vector indicating that a known defect occurs on execution path A, the similarity between the defect vector and the execution path vector corresponding to the test case is calculated to determine which path is more likely to be associated with the defect.
[0059] Further, based on the similarity calculation result, the reference test case is matched with the preset defect data to form a matching relationship, and the reference test case is divided into multiple test case groups according to the matching relationship, each group containing test cases with high similarity to specific defect data. For example, if the test case of path 1 has a high similarity with a preset defect data, and the test case of path 2 has a high similarity with another preset defect data, then the two test cases can be respectively classified into different test case groups.
[0060] Furthermore, the test cases in each test case group are simulated, that is, without actually running the program, their execution paths and possible outputs are simulated, and a weight is assigned to each test case path, that is, each node, based on the simulation results, such as path coverage, execution time, resource consumption, etc. During the simulation execution process, if a test case path covers more lines of code or triggers more boundary conditions, then the path may be assigned a higher weight.
[0061] Furthermore, each reference defect data also has a corresponding weight, indicating the importance of the defect. Test cases are screened based on the weight of the test case path and the weight of the reference defect data. Usually, test cases with high similarity to important defect data and high weight are selected for priority testing.
[0062] Specifically, the weights assigned to the test cases in each test case group after simulation execution and the weights corresponding to the reference defect data corresponding to each test case group are determined, so as to calculate the total weights corresponding to each test case in each test case group. The test cases with higher total weights are tested first. For example, if a test case group matches a very critical defect data, and a test case path in the test case group has a very high weight (because it covers the critical code logic), then this test case may be selected as a candidate for priority testing.
[0063] In one embodiment of the present application, a reference test case set is constructed based on each reference test case. Based on the isolation forest algorithm, the abnormal scores corresponding to each reference test case are determined, and the abnormal reference test cases in the reference test case set are screened out based on the abnormal scores. Feature extraction is performed on the paths of each reference test case after screening to obtain data feature values corresponding to each reference test case. Each data feature value is combined to obtain a data feature set corresponding to the reference test case set. Each data feature value is projected in the data feature set to obtain a first data feature vector corresponding to each data feature value. Based on the difference in data features, weights are assigned to different data features in the first data feature vector to obtain second data feature vectors corresponding to each test case.
[0064] Specifically, all relevant reference test cases are collected and organized into a set to obtain a reference test case set. For each test case, its feature values, such as execution time, resource consumption, etc., are extracted, and the abnormal score of each test case is calculated by presetting the isolation forest algorithm. A threshold is set according to the abnormal score, and the test cases with scores exceeding the threshold are regarded as abnormal and screened out. For example, the execution time and resource consumption of each test case are extracted as features, and their abnormal scores are calculated using the isolation forest algorithm. A threshold is set, such as the 95% quantile of the abnormal score, and it is found that the abnormal scores of 5 test cases exceed the threshold, so they are screened out from the reference test case set. Among them, the training process of the isolation forest algorithm training model in the embodiment of the present application is: the collected historical test cases are used as training sets, and the features of each historical test case in the training set are extracted, and key parameters, such as the number of trees, sample size, and the maximum number of features, are determined to perform model training.
[0065] Furthermore, for each test case after the exception is screened out, the execution path of each test case is analyzed to extract path-related feature values, such as path length, the number of specific nodes (such as function calls, conditional judgments), etc. The feature value of each test case is taken as a row, and the feature values of all test cases are combined into a matrix. This matrix is the data feature set, in which each column represents a feature and each row represents a test case. According to the preset feature weight table, a weight is assigned to each feature, wherein the preset feature weight table includes multiple different features, and also includes weight values corresponding to different features. The feature value of each test case is multiplied by its corresponding weight to obtain a weighted feature value, and the weighted feature values are combined into a second data feature vector.
[0066] For example, by querying the preset feature weight table, a higher weight is assigned to the path length because it directly affects the complexity and coverage of the test case, a medium weight is assigned to the number of function calls, and a lower weight is assigned to the conditional branch coverage. Then, the feature value of each test case is multiplied by its corresponding weight, and the results are combined into the second data feature vector.
[0067] Step 105: execute the reference test case, and when the test coverage meets a preset coverage threshold, determine that the reference test case is a required test case.
[0068] In one embodiment of the present application, a reference test case is executed to obtain test coverage. The test coverage is compared with a preset coverage threshold, and when the test coverage is less than the preset coverage threshold, the path not covered by the test is determined. For the path not covered by the test, a new test case is generated, and a new reference test case is generated based on the new test case and the reference test case to perform test case generation iteration. The new reference test case is executed until the test coverage corresponding to the new reference test case is greater than the preset coverage threshold, and the iterative process is terminated to obtain the required test case.
[0069] Specifically, execute the reference test case and record the execution result. Obtain how much code has been executed to obtain the test coverage. The embodiment of the present application is provided with a coverage threshold, which represents the minimum test coverage expected to be achieved. Compare the actual test coverage with this threshold. If the test coverage is lower than the preset threshold, then some code or paths are not covered by the test case. Identify these uncovered code paths, and generate new test cases for these uncovered paths. Merge the newly generated test cases with the original reference test cases to form a new reference test case set, and repeat the above steps (starting from executing the new reference test case) until the test coverage corresponding to the new reference test case set reaches or exceeds the preset coverage threshold. When the test coverage meets the requirements, stop the iterative process. At this time, the test case set obtained is the required test case, which can better cover the code and logic path of the software.
[0070] Traditional manual writing of test cases requires a lot of manpower and time costs, and is prone to errors. However, the embodiment of the present application reduces these costs by automatically generating test cases. At the same time, since the generation of test cases is based on code analysis, it can be automatically updated as the code changes, reducing maintenance costs. At the same time, the embodiment of the present application can span multiple programming languages and platforms to achieve a unified testing strategy. This avoids the cumbersome process of writing test codes separately for each programming language and target platform, thereby significantly improving testing efficiency.
[0071] Figure 2 A unit test case generation example diagram provided in an embodiment of the present application, such as Figure 2 As shown, take C++ code as an example:
[0072] / / subarray.cpp
[0073] bool subArray(int* arr, int size, int&result) {
[0074] if(arr = nullptr || size == 0) {
[0075] return false;
[0076] }
[0077] for (int i = 0; i <size; ++i) {
[0078] result -= arr[i];
[0079] }
[0080] return true;
[0081] }
[0082] First, perform lexical analysis and grammatical analysis on the above code and generate an intermediate representation. The command is as follows:
[0083] clang -S -emit-llvm . / subarray.cpp –o subarray.ll
[0084] Then we start to analyze the code of the intermediate representation and find out that the function currently contained is subarray, and its parameters are pointer, integer and reference respectively.
[0085] Based on the code analysis, the function is analyzed for boundary conditions and exceptions, and its pointer is judged to be null, array size, etc., and its return value is judged. At the same time, different execution paths of the program and the constraints of the paths are explored. All the above information is stored in the object instantiated by the unit test class to generate unit test cases. The command to specify the code analysis pass is as follows:
[0086] opt –codeanalysis subarray.ll
[0087] Finally, combined with the test framework specified by the user, here we take the Google test framework as an example to output specific test code. The generated test code is as follows:
[0088] TEST(SumArrayTest, SumsValidArray) {
[0089] int array[] = {1, 2, 3, 4, 5};
[0090] int result = 0;
[0091] EXPECT_TRUE(sumArray(array, 5, result));
[0092] EXPECT_EQ(result, 15);
[0093] }
[0094] TEST(SumArrayTest, HandlesNullPointer) {
[0095] int* array = nullptr;
[0096] int result = 0;
[0097] EXPECT_FALSE(sumArray(array, 5, result));
[0098] EXPECT_EQ(result, 0);
[0099] }
[0100] Execute the unit test code and generate a test coverage report. The above test coverage is 80%.
[0101] Analyze the test coverage. If the test coverage threshold is set to 90% and the actual test coverage is insufficient, iterate again and add unit test cases with parameter size of 0. Analyze the code again according to the test coverage report and generate unit test cases as follows:
[0102] TEST(SumArrayTest, HandlesEmptyArray) {
[0103] int array[0];
[0104] int result = 0;
[0105] EXPECT_FALSE(sumArray(array, 0, result));
[0106] EXPECT_EQ(result, 0);
[0107] }
[0108] The generated test coverage is 100%, which meets the requirements. So far, the method of automatically generating unit tests based on LLVM is completed.
[0109] Figure 3 A schematic diagram of the structure of a unit test case generation device provided in an embodiment of the present application. Figure 3As shown, a unit test case generating device 200 includes: at least one processor 201; and a memory 202 that is communicatively connected to the at least one processor 201; wherein the memory 202 stores instructions that can be executed by the at least one processor 201, and the instructions are executed by the at least one processor 201 so that the at least one processor 201 can: parse the acquired source code through the compiler LLVM to generate an intermediate representation; perform code analysis and boundary analysis on the intermediate representation to obtain a unit to be tested; replace input variables with symbolic values, and collect path constraints for different paths corresponding to the unit to be tested through the symbolic values; obtain test case information based on the collected path constraints, determine a corresponding test framework based on the test case information, and generate a reference test case; execute the reference test case, and determine that the reference test case is a required test case when the test coverage meets a preset coverage threshold.
[0110] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, wherein the computer executable instructions are configured to: parse an acquired source code through a compiler LLVM to generate an intermediate representation; perform code analysis and boundary analysis on the intermediate representation to obtain a unit to be tested; replace input variables with symbolic values, and collect path constraints for different paths corresponding to the unit to be tested through the symbolic values; obtain test case information based on the collected path constraints, determine a corresponding test framework based on the test case information, and generate a reference test case; execute the reference test case, and determine that the reference test case is a required test case when the test coverage meets a preset coverage threshold.
[0111] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0112] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application.
Claims
1. A method for generating unit test cases, characterized in that: The method comprises: The obtained source code is parsed through the compiler LLVM to generate an intermediate representation; Performing code analysis and boundary analysis on the intermediate representation to obtain a unit to be tested; Replace the input variable with a symbolic value, and collect path constraint conditions for different paths corresponding to the unit to be tested respectively through the symbolic value; Based on the collected path constraint conditions, test case information is obtained, a corresponding test framework is determined based on the test case information, and a reference test case is generated; Execute the reference test case, and determine that the reference test case is a required test case when the test coverage meets a preset coverage threshold; The performing code analysis and boundary analysis on the intermediate representation to obtain a unit to be tested specifically includes: Traversing the function definition nodes in the intermediate representation, and recording the function definition information corresponding to each of the function definition nodes; wherein the function definition information includes at least one of a name, a parameter, and a return type; Constructing a control flow graph for the intermediate representation, so as to perform control flow analysis through the control flow graph and obtain data flow information; Determining code data with boundaries in the intermediate representation and determining boundary conditions; wherein the code data with boundaries includes at least one of a loop statement, a conditional statement, an array access, and a pointer dereference; Based on the function definition information, the data flow information and the boundary conditions, the unit to be tested is obtained; wherein the unit to be tested includes at least one of a key function, a boundary condition and an abnormal path; The step of replacing the input variable with a symbolic value and collecting path constraint conditions for different paths corresponding to the unit to be tested respectively through the symbolic value specifically includes: Determine a control flow graph corresponding to the unit to be tested; replacing input variables with symbolic values, and causing the symbolic values to traverse the control flow graph to generate different execution paths based on conditional branches in the control flow graph; Based on the branch points of the execution path, a corresponding conditional expression is generated, and the conditional expression is used as a branch point constraint condition; Based on the multiple branch point constraints corresponding to the unit to be tested, a path constraint corresponding to the test unit is obtained; The executing the reference test case, and determining that the reference test case is a required test case when the test coverage meets a preset coverage threshold, specifically includes: Execute the reference test case to obtain test coverage; Comparing the test coverage with the preset coverage threshold, and determining the paths not covered by the test when the test coverage is less than the preset coverage threshold; Generate a new test case for the path not covered by the test, and generate a new reference test case based on the new test case and the reference test case to perform test case generation iteration; Execute the new reference test case until the test coverage rate corresponding to the new reference test case is greater than the preset coverage rate threshold, end the iteration process, and obtain the required test case; After determining the corresponding test framework based on the test case information and generating a reference test case, the method further includes: Vectorizing the execution path corresponding to the reference test case; Performing similarity calculation on the vectorized execution path and the preset defect data vector, so as to determine the reference defect data matching the reference test case based on the similarity result; Based on the matching relationship, the reference test cases are divided into a plurality of test case groups; Based on each of the test case groups, each test case path is taken as a node, the test case is simulated and executed on the test case path, and weights are assigned to the nodes based on the simulation results; Based on the weight distribution result and the weights respectively corresponding to the reference defect data, the reference test cases are screened; The vectorizing the execution path corresponding to the reference test case specifically includes: Based on each of the reference test cases, construct a reference test case set; Based on the isolation forest algorithm, determining the abnormal scores corresponding to the reference test cases, and filtering out the abnormal reference test cases in the reference test case set based on the abnormal scores; Performing feature extraction on the paths of the reference test cases after screening to obtain data feature values corresponding to the reference test cases; Combining the data feature values to obtain a data feature set corresponding to the reference test case set; Projecting each of the data feature values on the data feature set to obtain first data feature vectors corresponding to each of the data feature values; Based on the difference in data features, weights are assigned to different data features in the first data feature vector to obtain second data feature vectors corresponding to each of the test cases.
2. A method for generating unit test cases according to claim 1, characterized in that: The obtaining of test case information based on the collected path constraint conditions, determining a corresponding test framework based on the test case information, and generating a reference test case specifically includes: Inputting the path constraint conditions corresponding to the unit to be tested into a constraint solver to obtain input values for triggering different execution paths; Obtain the test case information based on the input value; Matching the test case information with a preset test framework reference table to determine a reference test framework; wherein the preset test framework reference table includes a plurality of test frameworks, and also includes integration conditions corresponding to the plurality of test frameworks; the integration conditions include at least one of integration capability, language type required for integration, extensibility, and maintainability; In the reference test framework, the reference test case is generated according to the test case information.
3. A method for generating unit test cases according to claim 1, characterized in that: The obtained source code is parsed by the compiler LLVM to generate an intermediate representation, which specifically includes: Preprocessing the acquired source code through the compiler LLVM; wherein the preprocessing includes macro definition replacement and header file import; The preprocessed source code is segmented, and word sequences are combined based on the segmented source code to construct a syntax tree; The syntax tree is traversed from top to bottom by a code generator to generate an intermediate representation.
4. A unit test case generation device, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1 to 3.
5. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions can execute the method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Test case generation system based on dynamic symbolic execution
CN110515856A
Test case generation method based on dynamic symbolic execution
CN110515857A
Unit test automatic execution method based on test case automatic generation algorithm
CN110543421A
Unit test method and system, electronic equipment and storage medium
CN114281709A
Test case generation method and device and electronic equipment
CN117632721A
