A method, device, and storage medium for generating test cases.

Abstract

This invention discloses a method, device, and storage medium for generating test cases. The method includes: constructing an abstract syntax tree (AST) from the source code to be tested; the source code contains multiple methods to be tested; collecting the execution results of the methods under constraints in each branch of the AST to obtain the original execution path; reducing the original execution path according to a set optimization metric to obtain a target execution path that covers the constraints; extracting the context information of the methods from the target execution path; constructing target prompts based on the context information; and inputting the target prompts into a preset language model to generate test cases. This embodiment searches for target execution paths that cover constraints, ensuring the accuracy of the test objects and effectively reducing computational load. It uses the context information of the methods in the target execution path to construct target prompts, guiding the generation of test cases in the language model and constructing high-quality test cases.

Description

Technical Field

[0001] This invention relates to the technical field of software testing, and in particular to a method, device and storage medium for generating test cases. Background Technology

[0002] During the software development process, the software is tested to discover and fix errors and defects, ensuring that the final product meets user needs and functions properly.

[0003] Currently, search-based software testing (SBST) is mainly used to generate test cases, which are then invoked to test the software.

[0004] However, SBST relies on a search algorithm to traverse the possible input space. But for complex software structures, such as deeply nested conditional statements and multi-level loops, the effective coverage of the execution path by the search algorithm is incomplete, resulting in low quality test cases and omissions in verifying the various branches and boundary conditions of the software. Summary of the Invention

[0005] In view of this, the present invention provides a method, device and storage medium for generating test cases, so as to improve the quality of test cases.

[0006] A first aspect of the present invention provides a method for generating test cases, comprising:

[0007] An abstract syntax tree is constructed from the source code to be tested; the source code contains multiple methods to be tested.

[0008] The execution results of the method under constraints are collected in each branch of the abstract syntax tree to obtain the original execution path;

[0009] The original execution path is reduced according to the set optimization scale to obtain the target execution path that covers the constraints.

[0010] Extract the context information of the method from the target execution path;

[0011] Target prompts are constructed based on the context information;

[0012] The target prompt words are input into a preset language model to generate test cases.

[0013] A second aspect of the present invention provides a test case generation apparatus, comprising:

[0014] An abstract syntax tree building module is used to construct an abstract syntax tree from the source code to be tested; the source code contains multiple methods to be tested.

[0015] The original execution path collection module is used to collect the execution results of the method under constraints in each branch of the abstract syntax tree to obtain the original execution path;

[0016] The target execution path reduction module is used to reduce the original execution path according to the set optimization scale to obtain a target execution path that covers the constraints.

[0017] A context information extraction module is used to extract the context information of the method from the target execution path;

[0018] The target prompt word construction module is used to construct target prompt words based on the context information;

[0019] The test case generation module is used to input the target prompt words into a preset language model to generate test cases.

[0020] A third aspect of the present invention provides an electronic device, the electronic device comprising:

[0021] At least one processor; and

[0022] A memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the test case generation method as described in the first aspect above.

[0024] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the test case generation method as described in the first aspect above.

[0025] A fifth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the test case generation method as described in the first aspect above.

[0026] In this embodiment, an abstract syntax tree (AST) is constructed from the source code to be tested. The source code contains multiple methods to be tested. The execution results of the methods under constraints are collected from each branch of the AST to obtain the original execution path. The original execution path is reduced according to a set optimization metric to obtain the target execution path that covers the constraints. Contextual information of the methods is extracted from the target execution path. Target prompts are constructed based on the contextual information. The target prompts are input into a pre-defined language model to generate test cases. This embodiment searches for target execution paths that cover the constraints, ensuring the accuracy of the test object and effectively reducing computational load. Using the contextual information of the methods in the target execution path to construct target prompts guides the generation of test cases in the language model, allowing the language model to fully understand the information of the target execution path and construct high-quality test cases.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of a test case generation method provided in Embodiment 1 of the present invention.

[0030] Figure 2 This is a flowchart of a test case generation method provided in Embodiment 2 of the present invention.

[0031] Figure 3 This is a schematic diagram of the structure of a test case generation device provided in Embodiment 3 of the present invention.

[0032] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate so that the embodiments of the invention described herein can cover implementations in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] Example 1

[0036] See Figure 1 This diagram illustrates a flowchart of a test case generation method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where execution paths in source code are reduced and test cases are constructed using a Language Model (LM) based on the context information of the execution path. This method can be executed by a test case generation device, which can be implemented in hardware and / or software. The test case generation device can be configured in an electronic device that has a software testing platform installed, such as PingCode, TestLink, Kiwi TCMS, Squash™, FitNesse, Tuleap, Robot Framework, SpecFlow, TestMaster, Nitrate, etc. Figure 1 As shown, the method includes:

[0037] Step 101: Construct an abstract syntax tree for the source code to be tested.

[0038] In this embodiment, the source code of the software to be tested can be input into the software testing platform, wherein the source code contains multiple methods to be tested.

[0039] In a software testing platform, the syntax structure of the source code can be parsed using appropriate parsers such as Tree-sitter, JavaScriptParser, and Babylon, based on the programming language of the source code, thereby constructing an Abstract Syntax Tree (AST). An AST represents the syntax structure of a programming language in a tree-like form, and each node in the AST represents a structure in the source code.

[0040] Taking Tree-sitter as an example, it can use the syntax definition file of a specific programming language (such as CFG (Context-Free Grammar)) to parse the source code. Each programming language has its corresponding syntax file, which defines the syntax rules of the language (such as keywords, syntax structure, etc.) and can be parsed by Tree-sitter into internal data structures for generating AST.

[0041] The Tree-sitter parsing function is a variant of the LR(1) parsing algorithm, which can handle complex language structures such as left recursion and ambiguity on a single scan basis.

[0042] The source code string is passed to the Tree-sitter parsing function, which parses the source code according to the defined syntax rules and builds an abstract syntax tree.

[0043] Step 102: Collect the execution results of the methods under constraints in each branch of the abstract syntax tree to obtain the original execution path.

[0044] In the initial stage of generating test cases, path collection can be performed in the abstract syntax tree. This involves traversing the structure of the source code in the abstract syntax tree to extract all possible execution paths from the source code, which are denoted as original execution paths. Each original execution path represents the execution flow of the source code under certain constraints.

[0045] In practical implementation, a depth-first traversal (DFFS) can be performed on the abstract syntax tree to identify branching structures in the source code, such as if-else statements and switch-case statements.

[0046] During the depth-first traversal, the constraints (such as cond1, cond2, etc.) of each node in each branch structure are recorded in sequence, as well as their execution results (True or False), thereby generating a possible execution path, which is recorded as the original execution path.

[0047] This process of traversing each branch structure continues until the entire source code has been traversed. In this way, all possible original execution paths are generated and collected into the path set P.

[0048] For each original execution path p i All of them can be represented as a conditional sequence (e.g., p). i = (cond1 = True, cond2 = False, cond3 = True)). At this point, the path set P contains all possible original execution paths, with a quantity of 2. n There are n constraints, where n is the number of constraints.

[0049] For example, assuming there is a branch structure check_file_status(path:PATH)→str, the process of traversing the original execution path is as follows:

[0050] S1. The first method statement, resolve_path(path), is accessed. Since it does not involve constraints, no information is logged.

[0051] S2. Enter the first if statement and record the constraints of path.exists() to ensure that the current path is executable on the abstract syntax tree. Iterate to the return'missing' statement and record the path that returns 'missing' if path.exists() is false.

[0052] S3. Under the condition that path.exists() is true, continue to traverse and visit the second if statement, record the constraint that path.is_readable() is true, and record the path that returns 'readable'.

[0053] S4. Continue iterating until the final `return 'unreadable'` statement is reached. At this point, record the path where `path.exists()` is true but `path.is_readable()` is false. Generate the original execution path using the collected paths and return values ​​(i.e., execution results). This path is used to generate test hints, specifying constraints and execution results.

[0054] Step 103: Reduce the original execution path according to the set optimization scale to obtain the target execution path that covers the constraints.

[0055] In this embodiment, an optimization scale can be set, and all original execution paths in the path set can be reduced according to the optimization scale. Reduction is an optimization step, the purpose of which is to reduce the number of original execution paths to be tested. The remaining original execution paths are recorded as target execution paths. In this way, the amount of computation in testing can be reduced, providing a data foundation for optimizing test cases. At the same time, it ensures that each target execution path can cover unique constraints.

[0056] Among them, the optimization scale is an adjustable parameter in the reduction. For a given number of original execution paths, the larger the optimization scale, the more target execution paths there are after reduction. Conversely, the smaller the optimization scale, the fewer target execution paths there are after reduction. This is used to adjust the number of target execution paths after reduction.

[0057] In the specific implementation, the set of minimizing constraints minconstraints and the set of minimizing paths minpaths can be initialized, and both the set of minimizing constraints minconstraints and the set of minimizing paths minpaths are empty sets.

[0058] During the traversal of each original execution path, a new original execution path p is extracted from the path set P using random methods. Each constraint condition path_constraints is extracted from the current original execution path p. The constraints path_constraints of the current original execution path p are compared with the constraints path_constraints in the minimized constraint set minconstraints to determine whether the minimized constraint set minconstraints contains the constraints path_constraints of the current original execution path p.

[0059] If at least one constraint path_constraints in the current original execution path p is different from all constraints path_constraints in the minimization constraint set minconstraints, that is, if the current original execution path p has at least one new constraint path_constraints, then at least one new constraint path_constraints in the current original execution path p can be added to the minimization constraint set minconstraints, and the current original execution path p can be added to the minimization path set minpaths.

[0060] If all constraints path_constraints in the current original execution path p are the same as those in the minimized constraint set minconstraints, meaning there are no new constraints path_constraints in the current original execution path p, then the number of other original execution paths that contain the constraints path_constraints in the current original execution path p can be counted in the minimized path set minpaths.

[0061] Compare this quantity with the set optimization scale, which is an integer greater than or equal to 0.

[0062] If the number is greater than the set optimization scale, then the current original execution path p is ignored.

[0063] If the number is less than or equal to the set optimization scale, then the current original execution path p is added to the set of minimized paths minpaths.

[0064] If all the original execution paths in the path set P have been traversed, then the original execution path in the constraint set minpaths is determined as the target execution path. At this time, each original execution path in the constraint set minpaths covers at least one unique constraint condition.

[0065] Reduction describes the process of minimizing the original execution path, with the goal of finding the minimum set of paths that satisfy the minimum constraints within a given set of paths.

[0066] Therefore, reduction can be expressed using the following mathematical expression:

[0067]

[0068] Where minpaths is the final set of minimized constraints, paths is the set of paths, p is the original execution path in the set of paths, splitConstraints is the set of minimized constraints, minconstraints is the set of minimized constraints that has already been included, and c is the condition for existence.

[0069] For n constraints, the number of target execution paths after reduction, |M|, is at most n+1, that is, |M|≤n+1, which can effectively reduce the number of target execution paths.

[0070] Step 104: Extract the context information of the method from the target execution path.

[0071] In this embodiment, the contextual information of the method under test is analyzed in depth within the target execution path. The contextual information is the basis for constructing effective prompt words, enabling the language model to fully understand the method under test and ensuring that the prompt words can fully guide the language model to generate test cases that meet expectations.

[0072] In the specific implementation, a depth-first traversal is performed on the abstract syntax tree to locate the node where the method to be tested is located in the target execution path. The signature information of the method is extracted from the node of the target execution path as context information. The signature information includes information such as name, parameter list, parameter type and return type.

[0073] The meaning of methods with different parameter types is different, so the context information of the corresponding method can be expanded from the nodes of the target execution path according to the parameter type.

[0074] For example, parameter types include basic data types, user-defined types, and complex data types.

[0075] Basic data types include int, float, etc., which can usually be used directly and recognized by the language model.

[0076] If the parameter type is a user-defined type, such as a class or struct, then the information of the user-defined type's attributes (member variables) and definitions is read from the nodes of the target execution path as context information so that the language model can correctly generate test cases related to the user-defined type.

[0077] If the parameter type is a complex data type, such as a pointer, array, or template class, then the basic data type pointed to by the complex data type is parsed as context information. Additionally, the usage pattern of the complex data type is read from the nodes of the target execution path using methods such as type inference and recursion as context information.

[0078] For example, for template classes in C++, their instantiation information can be parsed to obtain usage patterns in order to determine the specific template parameter types, and the specific manifestations of these template parameter types in different instantiation scenarios can be considered.

[0079] Furthermore, this embodiment constructs test cases through a multi-round iteration pattern. If the current iteration is not the first iteration, the test results recorded when the source code was tested using the test cases in the previous iteration can be queried, and the test results can be expanded into the context information of the method to be tested in the current iteration.

[0080] This embodiment implements a context expansion and feedback mechanism based on the test results of test cases. This not only enriches the background information of the language model, but also guides the language model to refer to successful and unsuccessful cases in the test results of previous test cases when generating new test cases. It uses successful experiences, avoids repeating errors, and avoids getting stuck in infinite loops, thereby generating more effective and comprehensive test cases and ensuring that the final test cases of the iteration can continuously approach the expected test results.

[0081] Step 105: Construct target prompts based on contextual information.

[0082] In this embodiment, prompt words can be constructed based on the context information of the method to be tested, ensuring that all collected context information is reasonably and fully expressed in the prompt words, thereby helping the language model generate accurate test cases.

[0083] In the specific implementation, it is possible to query the prompt templates that match the information types set in the source code and the context information; among them, the information types include parameter types in the signature information, test results, etc.

[0084] Contextual information is written into the prompt template according to the information type, resulting in structured original prompt words. This ensures that the contextual information of each information type is appropriately expressed. For example, for a method containing a template class, the original prompt word clearly indicates the type of the template parameter; for complex data types, the original prompt word clearly indicates the object pointed to by the pointer, the size and contents of the array, etc., to help the language model correctly understand and generate reasonable test cases.

[0085] If the context information for each information type has been used to construct the original prompt words, then all the original prompt words can be combined to form the target prompt word.

[0086] The prompt template contains information about the specific test scenario. Writing contextual information into the prompt template can combine the contextual information with the specific test scenario, instructing the language model to behave when generating test cases.

[0087] For example, for signature information of certain complex data types, the target prompt should clearly specify how to initialize them and how to call the methods they provide in the test case.

[0088] For certain user-defined signature types, the target prompt should explicitly indicate which attributes to initialize, which methods to call, and so on.

[0089] Step 106: Input the target prompt words into the preset language model to generate test cases.

[0090] In this embodiment, the target prompt word can be input into a preset language model, especially an LLM (Large Language Model), and the language model generates test cases according to the guidance of the target prompt word.

[0091] In this embodiment, an abstract syntax tree (AST) is constructed from the source code to be tested. The source code contains multiple methods to be tested. The execution results of the methods under constraints are collected from each branch of the AST to obtain the original execution path. The original execution path is reduced according to a set optimization metric to obtain the target execution path that covers the constraints. Contextual information of the methods is extracted from the target execution path. Target prompts are constructed based on the contextual information. The target prompts are input into a pre-defined language model to generate test cases. This embodiment searches for target execution paths that cover the constraints, ensuring the accuracy of the test object and effectively reducing computational load. Using the contextual information of the methods in the target execution path to construct target prompts guides the generation of test cases in the language model, allowing the language model to fully understand the information of the target execution path and construct high-quality test cases.

[0092] Example 2

[0093] See Figure 2 The diagram illustrates a flowchart of a test case generation method provided in Embodiment 2 of the present invention. This embodiment adds a process for optimizing test cases based on the aforementioned embodiments. Figure 2 As shown, the method includes:

[0094] Step 201: Construct an abstract syntax tree for the source code to be tested.

[0095] The source code contains several methods that need to be tested.

[0096] Step 202: Collect the execution results of the methods under constraints in each branch of the abstract syntax tree to obtain the original execution path.

[0097] Step 203: Reduce the original execution path according to the set optimization scale to obtain the target execution path that covers the constraints.

[0098] Step 204: Extract the context information of the method from the target execution path.

[0099] Step 205: Construct target prompts based on contextual information.

[0100] Step 206: Input the target prompt words into the preset language model to generate test cases.

[0101] Step 207: Use test cases to test the source code and obtain the test results.

[0102] Each generated test case is executed, the source code is tested, and the test results are recorded in real time. During the execution of the test cases, all possible exceptions, errors, and method return values ​​are captured to ensure the reliability and accuracy of the test cases.

[0103] Furthermore, a comprehensive exception handling mechanism is set up during the execution of test cases. If a test case calls a non-existent method or uses an incorrect parameter type, the test case is recorded as a failure, and the exception type and corresponding stack information are recorded in detail.

[0104] Step 208: Calculate multiple original evaluation metrics for the test cases based on the test results.

[0105] In this embodiment, the automated testing tools provided by the software testing platform can be invoked to analyze the test results, thereby calculating multiple raw evaluation indicators for the test cases.

[0106] For example, the original evaluation metrics include successful execution rate, successful call rate, line coverage, and branch coverage.

[0107] The success rate is the percentage of test cases that are successfully executed on the first run out of the total number of test cases. The success rate reflects the effectiveness of the generated test cases on their first run.

[0108] The success rate can be expressed as:

[0109]

[0110] Where Pass@1 represents the success rate, and T i Let N be the i-th test case and N be the number of test cases. Exec(T) i ) is test case T i The execution result, if test case T i If the execution is successful and without errors, then Exec(T) i ) Returns 1, otherwise Exec(T) i Returns 0.

[0111] The success rate is the percentage of test cases that are successfully invoked and executed without errors out of the total number of test cases, reflecting the accuracy of the generated test cases.

[0112] The success rate of a call can be expressed as:

[0113]

[0114] Where FM Call@1 represents the success rate of calls, and T i Let N be the i-th test case and N be the number of test cases. Exec(T) i ) is test case T i The execution result, if test case T i If the execution is successful and without errors, then Exec(T) i ) Returns 1, otherwise Exec(T) iCall(Target, T) returns 0. i ) is test case T i Does the function of the method under test call correctly? If the call is correct, call(Target, T) i Returns 1 if the value is not specified, otherwise Call(Target, T) i Returns 0.

[0115] Line coverage is defined as the proportion of lines of code executed when a test case is executed to the total number of lines of code. It can be used to measure the depth of coverage of the source code by the test case.

[0116] Row coverage can be defined as follows:

[0117]

[0118] Where Line Coverage is the line coverage rate, L i Let M be the number of lines of code in the i-th line, and M be the total number of lines in the source code. Cover(L) i ) indicates whether the test case covers line L of code. i If a function is covered by at least one test case, then Cover(L) i Returns 1 if Cover(L) is not found otherwise. i Returns 0.

[0119] Branch coverage is defined as the proportion of branch structures covered by test cases to the total number of branches. It can be used to evaluate whether test cases adequately cover all possible execution paths in the source code.

[0120] Branch coverage can be expressed as:

[0121]

[0122] Where Branch Coverage is the branch coverage rate, B j Let B be the j-th branch structure, and B be the total number of branch structures. Cover(B) j ) Whether the test cases cover the branch structure B j The function, if covered by at least one test case, is called Cover(B). j Returns 1 if Cover(B) is not found otherwise. j Returns 0.

[0123] Of course, the above-mentioned original evaluation indicators are only examples. When implementing this embodiment, other original evaluation indicators can be set according to the actual situation, such as defect density, defect closure speed, average repair time, performance indicators (such as response time, throughput, etc.), etc. This embodiment does not limit this. In addition, besides the above-mentioned original evaluation indicators, those skilled in the art can also use other original evaluation indicators according to actual needs, and this embodiment does not limit this either.

[0124] Step 209: Integrate multiple original evaluation indicators into target evaluation indicators.

[0125] In this embodiment, multiple original evaluation indicators can be fused into a target evaluation indicator using linear or nonlinear methods.

[0126] For example, the product of the successful execution ratio and the first weight, the product of the successful call ratio and the second weight, the product of the line coverage and the third weight, and the product of the branch coverage and the fourth weight are added together to obtain the target evaluation index, which can comprehensively measure the correctness and coverage of test cases.

[0127] Therefore, the target evaluation index can be expressed as:

[0128] Overall Quality

[0129] =α·Pass@1+β·FM Call@1+γ·Line Coverage

[0130] +δ·Branch Coverage

[0131] Among them, Overall Quality is the target evaluation index, Pass@1 is the successful execution rate, FM Call@1 is the successful call rate, Line Coverage is the line coverage rate, Branch Coverage is the branch coverage rate, α is the first weight, β is the second weight, γ is the third weight, δ is the fourth weight, and α+β+γ+δ=1.

[0132] Step 210: If the target evaluation index is less than or equal to the first threshold, and the number of test cases generated in the current iteration is less than or equal to the second threshold, then adjust the optimization scale upward according to some of the original evaluation indices and return to step 203.

[0133] In this embodiment, the target evaluation index can be compared with a preset first threshold (e.g., 0.8), and the number of test cases generated in the current iteration can be counted and compared with a preset second threshold (e.g., 3).

[0134] In addition to helping improve the accuracy of test cases, the second threshold can also prevent unresolved loops from occurring during the test case generation process.

[0135] If the target evaluation metric is less than or equal to the first threshold, and the number of test cases generated in the current iteration is less than or equal to the second threshold, then it can be decided whether to increase the optimization scale based on the situation of some original evaluation metrics. Steps 203 to 209 are then executed to enter the next iteration. At this time, more original execution paths can be included as target execution paths, and more detailed contextual information or more explicit constraints can be added to guide the language model to generate more accurate test cases.

[0136] For example, if the row coverage is less than or equal to the third threshold and the branch coverage is less than or equal to the fourth threshold, it indicates that the coverage of the test cases is low, and the optimization scale can be increased, such as setting the sum of the optimization scale and the preset step size as the new optimization scale.

[0137] At this point, paths of code that are not fully covered can be identified. For these paths, more precise target hints can be constructed, or more diverse test cases can be generated through iterative optimization to improve coverage.

[0138] Step 211: If the target evaluation index is greater than the first threshold, or the number of test cases generated in the current iteration is greater than the second threshold, then the test case construction is considered complete.

[0139] If the target evaluation metric is greater than the first threshold, or if the number of test cases generated in the current iteration is greater than the second threshold, then the test cases can be determined to be completed. The test cases are then output to the testers for review and correction. Once the review and correction are completed, the test cases are used to test the source code.

[0140] Example 3

[0141] See Figure 3 The diagram illustrates the structure of a test case generation device according to Embodiment 3 of the present invention. Figure 3 As shown, the device includes:

[0142] Abstract syntax tree construction module 301 is used to construct an abstract syntax tree from the source code to be tested; the source code contains multiple methods to be tested;

[0143] The original execution path collection module 302 is used to collect the execution results of the method under constraints in each branch of the abstract syntax tree to obtain the original execution path;

[0144] The target execution path reduction module 303 is used to reduce the original execution path according to the set optimization scale to obtain a target execution path that covers the constraints.

[0145] The context information extraction module 304 is used to extract the context information of the method from the target execution path;

[0146] Target prompt word construction module 305 is used to construct target prompt words based on the context information;

[0147] The test case generation module 306 is used to input the target prompt words into a preset language model to generate test cases.

[0148] In one embodiment of the present invention, the target execution path reduction module 303 includes:

[0149] The set initialization module is used to initialize the set of minimization constraints and the set of minimization paths;

[0150] The constraint comparison module is used to compare each constraint of the current original execution path with the constraint in the minimized constraint set during the traversal of each original execution path.

[0151] The difference processing module is used to add at least one constraint in the current original execution path to the minimized constraint set and add the current original execution path to the minimized path set if at least one constraint in the current original execution path is different from all constraints in the minimized constraint set.

[0152] The quantity statistics module is used to count the number of other original execution paths that contain the constraints in the current original execution path if all the constraints in the current original execution path are the same as the constraints in the minimized constraint set.

[0153] The path addition module is used to add the current original execution path to the minimized path set if the number is less than or equal to a set optimization scale.

[0154] The target execution path determination module is used to determine the original execution path of the minimized constraint set as the target execution path if all the original execution paths have been traversed.

[0155] In one embodiment of the present invention, the context information extraction module 304 includes:

[0156] The signature information extraction module is used to extract the signature information of the method from the nodes of the target execution path as context information; the signature information includes parameter types;

[0157] The parameter type expansion module is used to expand the context information of the method from the nodes of the target execution path according to the parameter type.

[0158] In one embodiment of the present invention, the parameter type includes basic data types, user-defined types, and complex data types;

[0159] The parameter type expansion module includes:

[0160] A custom processing module is used to read the attributes and definitions of the user-defined type from the nodes of the target execution path as context information if the parameter type is the user-defined type.

[0161] The complex processing module is configured to, if the parameter type is the complex data type, parse the basic data type pointed to by the complex data type as context information, and read the usage pattern of the complex data type from the nodes of the target execution path as context information.

[0162] In one embodiment of the present invention, the context information extraction module 304 further includes:

[0163] The test result query module is used to query the test results recorded when the source code is tested using the test cases described above;

[0164] The test result expansion module is used to expand the test results into context information of the method.

[0165] In one embodiment of the present invention, the target prompt word construction module 305 includes:

[0166] The prompt template query module is used to query prompt templates that match the information type of the context information set for the source code.

[0167] The original prompt word construction module is used to write the context information into the prompt template according to the information type to obtain the original prompt word;

[0168] The original prompt word composition module is used to compose all the original prompt words into target prompt words.

[0169] In one embodiment of the present invention, it further includes:

[0170] The source code testing module is used to test the source code using the test cases and obtain test results;

[0171] The original evaluation index calculation module is used to calculate multiple original evaluation indices for the test cases based on the test results.

[0172] The target evaluation index fusion module is used to fuse multiple original evaluation indices into a target evaluation index.

[0173] The optimization scale adjustment module is used to adjust the optimization scale according to a portion of the original evaluation index if the target evaluation index is less than or equal to a first threshold and the number of test cases generated in the current iteration is less than or equal to a second threshold, and then return to the target execution path reduction module 303.

[0174] The test case construction module is used to determine that the test case construction is complete if the target evaluation index is greater than a first threshold or the number of times the test case is generated in the current iteration is greater than a second threshold.

[0175] In one embodiment of the present invention, the original evaluation metrics include successful execution rate, successful call rate, row coverage rate, and branch coverage rate;

[0176] The target evaluation index fusion module includes:

[0177] The linear fusion module is used to add the products of the successful execution ratio and the first weight, the successful call ratio and the second weight, the row coverage rate and the third weight, and the branch coverage rate and the fourth weight to obtain the target evaluation index.

[0178] The optimization scale adjustment module includes:

[0179] The coverage adjustment module is used to adjust the optimization scale upward if the row coverage is less than or equal to a third threshold and the branch coverage is less than or equal to a fourth threshold.

[0180] The test case generation device provided in the embodiments of the present invention can execute the test case generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the test case generation method.

[0181] Example 4

[0182] See Figure 4This diagram illustrates a structural schematic of an electronic device according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0183] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0184] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0185] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the test case generation method.

[0186] In some embodiments, the test case generation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the test case generation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the test case generation method by any other suitable means (e.g., by means of firmware).

[0187] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0188] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0189] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0190] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0191] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0192] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0193] Example 5

[0194] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the test case generation method provided in any embodiment of this invention.

[0195] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0196] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0197] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating test cases, characterized in that, include: Construct an abstract syntax tree from the source code to be tested; The source code contains several methods to be tested; The execution results of the method under constraints are collected in each branch of the abstract syntax tree to obtain the original execution path; The original execution paths are reduced according to a set optimization scale to obtain target execution paths that cover the constraints. The optimization scale is an adjustable parameter in the reduction process. For a given number of original execution paths, the larger the optimization scale, the more target execution paths are obtained after reduction. Extract the context information of the method from the target execution path; Target prompts are constructed based on the context information; Input the target prompt words into a preset language model to generate test cases; The step of reducing the original execution path according to a set optimization metric to obtain a target execution path that covers the constraints includes: Initialize the set of constraints to be minimized and the set of paths to be minimized; During the traversal of each of the original execution paths, each constraint of the current original execution path is compared with the constraint in the minimized constraint set; If at least one of the constraints in the current original execution path is different from all the constraints in the minimized constraint set, then at least one of the constraints in the current original execution path is added to the minimized constraint set, and the current original execution path is added to the minimized path set. If all the constraints in the current original execution path are the same as the constraints in the minimized constraint set, then count the number of other original execution paths that contain the constraints in the current original execution path in the minimized path set. If the number is less than or equal to the set optimization scale, then the current original execution path is added to the set of minimized paths; If all the original execution paths have been traversed, then the original execution path of the minimized constraint set is determined as the target execution path.

2. The test case generation method according to claim 1, characterized in that, Extracting the context information of the method from the target execution path includes: The signature information of the method is extracted from the nodes of the target execution path as context information; the signature information includes parameter types. The context information of the method is expanded from the nodes of the target execution path according to the parameter type.

3. The method according to claim 2, characterized in that, The parameter types include basic data types, user-defined types, and complex data types; The step of expanding the context information of the method from the nodes of the target execution path according to the parameter type includes: If the parameter type is the user-defined type, then the attributes and definitions of the user-defined type are read from the nodes of the target execution path as context information; If the parameter type is the complex data type, then the basic data type pointed to by the complex data type is parsed as context information, and the usage pattern of the complex data type is read from the nodes of the target execution path as context information.

4. The test case generation method according to claim 2, characterized in that, The step of extracting the context information of the method from the target execution path further includes: Query the test results recorded when the source code is tested using the test cases described above; The test results are expanded to include contextual information about the method.

5. The test case generation method according to claim 1, characterized in that, The step of constructing target prompt words based on the context information includes: Query the prompt template set for the source code that matches the information type of the context information; The context information is written into the prompt template according to the information type to obtain the original prompt word; Combine all the original prompts into the target prompt.

6. The method for generating test cases according to any one of claims 1-5, characterized in that, Also includes: The source code was tested using the test cases to obtain test results; Based on the test results, multiple raw evaluation metrics are calculated for the test cases; The multiple original evaluation indicators are integrated into a target evaluation indicator; If the target evaluation index is less than or equal to the first threshold, and the number of times the test cases are generated in the current iteration is less than or equal to the second threshold, then the optimization scale is increased according to a portion of the original evaluation index, and the original execution path is reduced according to the set optimization scale to obtain the target execution path that covers the constraints. If the target evaluation index is greater than the first threshold, or if the number of times the test case is generated in the current iteration is greater than the second threshold, then the test case is determined to have been successfully constructed.

7. The test case generation method according to claim 6, characterized in that, The original evaluation metrics include successful execution rate, successful call rate, row coverage, and branch coverage. The process of integrating multiple original evaluation indicators into a target evaluation indicator includes: The target evaluation index is obtained by adding the products of the successful execution ratio and the first weight, the successful call ratio and the second weight, the row coverage rate and the third weight, and the branch coverage rate and the fourth weight. The step of adjusting the optimization scale according to a portion of the original evaluation indicators includes: If the row coverage is less than or equal to the third threshold, and the branch coverage is less than or equal to the fourth threshold, then the optimization scale is increased.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the test case generation method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for generating test cases as described in any one of claims 1-7.

Images