Test case generation methods, devices, storage media, and program products

Abstract

This application discloses a test case generation method, device, storage medium, and program product, relating to the field of software testing technology. It includes acquiring test cases to be optimized and code snippets to be tested, extracting and structuring boundary conditions in the code, constructing a pre-trained language model with structured prompt words to generate a test coverage analysis report, and filtering valid test cases, eliminating redundancy, and supplementing test cases corresponding to uncovered boundaries based on the report. This achieves automated generation of optimized test cases with complete boundary coverage, no redundancy, and controllable effectiveness. It overcomes the limitations of traditional tools in identifying deep boundaries and avoids the randomness and instability of simply relying on pre-trained language models to generate test cases, providing accurate and efficient testing support for code quality assurance.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and in particular to a test case generation method, device, storage medium, and program product. Background Technology

[0002] In the field of software engineering, code quality assurance is a core element in ensuring system stability and functional correctness. Unit testing, as the foundation of software testing, directly determines the completeness of the testing by its coverage.

[0003] Traditional testing tools can only calculate coverage at the syntactic level and cannot identify deep boundary assertions in the code (such as null values, extreme values, abnormal inputs, etc.), resulting in test cases failing to cover critical scenarios. On the other hand, using pre-trained language models for testing may lead to problems such as missing critical boundaries and generating redundant tests. Summary of the Invention

[0004] This application provides a test case generation method, device, storage medium, and program product to at least solve the problems in related technologies where test cases used by testing tools cannot cover key scenarios, pre-trained language models omit key boundaries, or generate redundant tests.

[0005] This application provides a test case generation method, comprising: obtaining test cases to be optimized and code snippets to be tested; performing static analysis on the code snippets to be tested to obtain boundary assertions corresponding to the code snippets; determining a structured boundary list based on the boundary assertions; determining structured prompt words based on the structured boundary list, the code snippets to be tested, and the test cases to be optimized; inputting the structured prompt words into a pre-trained language model to obtain at least one test coverage analysis report; and generating optimized test cases based on the test coverage analysis report and the test cases to be optimized.

[0006] This application also provides a test case generation device, comprising: a data acquisition module for acquiring test cases to be optimized and code snippets to be tested; a static analysis module for performing static analysis on the code snippets to be tested to obtain boundary assertions corresponding to the code snippets; a list determination module for determining a structured boundary list based on the boundary assertions; a prompt word determination module for determining structured prompt words based on the structured boundary list, the code snippets to be tested, and the test cases to be optimized; a report acquisition module for inputting the structured prompt words into a pre-trained language model to obtain at least one test coverage analysis report; and a test case generation module for generating optimized test cases based on the test coverage analysis report and the test cases to be optimized.

[0007] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described test case generation methods when executing the computer program.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described test case generation methods.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described test case generation methods.

[0010] This application provides a test case generation method, device, storage medium, and program product. By first acquiring the test cases to be optimized and the code snippets to be tested, extracting and structuring the boundary conditions in the code through static analysis, and then constructing a pre-trained language model with structured prompt words to generate a test coverage analysis report, the application finally selects valid test cases, eliminates redundancy, and supplements test cases corresponding to uncovered boundaries based on the report. This achieves automated generation of optimized test cases with complete boundary coverage, no redundancy, and controllable effectiveness. It overcomes the limitations of traditional tools in identifying deep boundaries and makes up for the randomness and instability of simply relying on pre-trained language models to generate test cases, providing accurate and efficient test cases for code quality assurance. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic diagram illustrating the application scenario of the test case generation method provided in this application;

[0013] Figure 2 A flowchart illustrating the process of generating test cases provided in the embodiments of this application;

[0014] Figure 3 This is a schematic diagram of the test case generation device provided in the embodiments of this application;

[0015] Figure 4 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0017] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0018] In the field of software testing, code quality assurance is the core pillar for ensuring system stability and functional correctness. Unit testing, as the cornerstone of the software testing system, directly determines the completeness and effectiveness of the testing work through its coverage, and is a key line of defense for preventing potential bugs and reducing later maintenance costs.

[0019] However, traditional testing tools have significant limitations: they can only count coverage at the syntax execution level, such as lines of code and branches, and cannot accurately identify deep semantic boundary conditions hidden in the code (such as null input, numerical extreme values, and abnormal scenario triggers). This results in test cases that are often superficial and fail to reach core business scenarios and abnormal branches. In addition, if pre-trained large language models (LLMs) are directly used to generate or evaluate test cases, problems such as missing critical boundaries, rampant redundant test cases, and lack of traceability of results are likely to occur, making it difficult to meet the reliability requirements of testing.

[0020] This section describes the specific application environment architecture or hardware architecture upon which the execution of the test case's execution method depends. (References) Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario for the test case generation method provided in this application. For example... Figure 1 As shown, the application scenarios of the test case generation method include: database 101 and server 102.

[0021] Database 101 may include a single database or a combination of multiple databases. For example, it may include one or more databases such as Oracle, MySQL, DRDS (Distributed Relational Database Service), and ES (Elasticsearch). This application does not impose any special restrictions on this.

[0022] Server 102 can be implemented using a single server or a cluster of multiple servers with more powerful processing capabilities and higher security. Where possible, it can also be replaced by a computer or laptop with strong computing power.

[0023] Database 101 and server 102 can communicate with each other. The server retrieves the test cases to be optimized and the code snippets to be tested from the database, performs static analysis on the code snippets to obtain the boundary assertions corresponding to the code snippets, determines a structured boundary list based on the boundary assertions, and combines the structured boundary list, the code snippets to be tested, and the test cases to be optimized to determine structured prompt words. The structured prompt words are input into a pre-trained language model to obtain a test coverage analysis report, and the test cases to be optimized are optimized based on the test coverage analysis report.

[0024] Figure 2 This is a schematic diagram illustrating the process of generating test cases provided in the embodiments of this application. For example... Figure 2 As shown, embodiments of this application provide a test case generation method, which is described in detail below:

[0025] S201: Obtain the test cases to be optimized and the code snippets to be tested.

[0026] In this step, the test cases and code snippets to be optimized are read from a specified location in the database, or they can be read from a location specified by the staff. Alternatively, test cases and code snippets to be optimized can be received from a data storage device.

[0027] S202: Perform static analysis on the code snippet to be tested to obtain the boundary assertions corresponding to the code snippet to be tested.

[0028] In this step, a static analysis method that does not execute code can be used. The code to be tested is processed by a syntax analysis tool. First, an abstract syntax tree of the code is built. Then, the tree structure is traversed to identify the target syntax nodes that carry boundary conditions. Finally, based on a preset boundary domain keyword library, the logical conditions in the nodes are transformed into standardized boundary assertions, completing the mapping from code to boundary rules.

[0029] For example, in the code snippet under test, for the comparison operation x < 100, the variable x is converted to be of type int, with an upper bound of 100 (excluding 100). For operations involving division 100 / x, the variable x (of type int) is converted to be non-zero. For the interval expression 0 <= x <= 100, the lower bound of variable x is 0 (inclusive), and its upper bound is 100 (inclusive). For expressions involving function calls, such as "len(list_1) > 0", the conversion results in a constraint on the core variable list_1 (of type list), namely "not empty". For the function definition "def add(x: int, y: int) -> int", the type information of parameters x and y can be directly extracted from its type annotation as basic boundary knowledge.

[0030] S203: Determine the list of structured boundaries based on the boundary assertions.

[0031] In this step, all boundary assertions are categorized and organized according to the preset data format specifications, aggregated by variable dimension, and supplemented with attributes such as variable type, boundary type (discrete / continuous / mixed), and value constraints to form a structured boundary list, ensuring that the boundary information can be directly called by the subsequent prompt word fusion step.

[0032] The structured boundary list can include the following for each boundary domain: boundary domain name, variable name, variable type (integer, floating point, character, etc.), value constraints (boundary values ​​of the variable), variable boundary conditions, boundary domain type, whether the variable contains boundary values, and whether the variable is required.

[0033] S204: Determine the structured prompt words based on the structured boundary list, the code snippet to be tested, and the test cases to be optimized.

[0034] In this step, based on the preset prompt word framework, the structured boundary list, code snippets to be tested, and test cases to be optimized can be used as core context information and combined with fixed fields to construct logically clear and uniformly formatted structured prompt words, providing clear basis and constraints for model reasoning.

[0035] S205: Input the structured prompt words into the pre-trained language model to obtain at least one test coverage analysis report.

[0036] In this step, the interface of the pre-trained language model can be called, and the constructed structured prompt words can be used as input to trigger the model to reason according to the task requirements and logical framework in the prompt words, and output a test coverage analysis report that conforms to the format specifications. At least one report can be obtained through multiple rounds or multiple model calls.

[0037] The test coverage analysis report may include the identifier of the test case, whether the test case exists in a certain domain unit test set, the name of the boundary domain to which it belongs, the proportion of boundary domains with test cases to all boundary domains, and the name of the boundary domain without test cases.

[0038] S206: Generate optimized test cases based on the test coverage analysis report and the test cases to be optimized.

[0039] In this step, the test coverage analysis report can be used as a basis to first screen, deduplicate, and cross-validate the original test cases to be optimized to obtain the test coverage analysis report to be used. Then, new test cases are added to the boundaries not covered in the test coverage analysis report to be used. The optimized test case set is obtained by combining and integrating them.

[0040] The process of supplementing new test cases for boundaries not covered in the test coverage analysis report can be described in steps S2061 to S2065 below.

[0041] As described in the above embodiments, this disclosure first obtains the test cases to be optimized and the code snippets to be tested, extracts and structures the boundary conditions in the code through static analysis, then constructs a structured prompt word input pre-trained language model to generate a test coverage analysis report, and finally filters effective test cases, eliminates redundancy, and supplements test cases corresponding to uncovered boundaries based on the report. This achieves the automated generation of optimized test cases with complete boundary coverage, no redundancy, and controllable effectiveness. It not only overcomes the limitation of traditional tools in being unable to identify deep boundaries, but also makes up for the randomness and instability of simply relying on pre-trained language models to generate test cases, providing accurate and efficient test cases for code quality assurance.

[0042] In one possible implementation, step S206 above, based on the test coverage analysis report and the test cases to be optimized, generates optimized test cases, including:

[0043] S2061: Based on the test coverage analysis report, identify the effective test cases among the test cases to be optimized.

[0044] This step may include extracting the attribute identifiers corresponding to each test case to be optimized from the test coverage analysis report, filtering out test cases that meet the validity criteria based on the attribute identifiers, and excluding invalid test cases.

[0045] S2062: Remove redundant use cases from the valid use cases to obtain the remaining use cases.

[0046] In this step, the data is grouped by boundary domain dimension, and the effective use cases within the same boundary domain are analyzed. Use cases that repeatedly cover the same constraint or are undifferentiated are eliminated, and representative use cases are retained to form a set of deduplicated remaining use cases.

[0047] S2063: Read the uncovered boundary areas in the test coverage analysis report.

[0048] In this step, you can read the specified fields (such as "missing_boundaries") of the test coverage analysis report, extract the list of boundary domains that are not covered by any test cases, i.e., uncovered boundary domains, and clarify the boundary ranges that need to be covered.

[0049] For example, assuming the "missing_boundaries" field in the test coverage analysis report contains ("B1", "B3"), after parsing this field, the two uncovered boundary regions, B1 (where the variable numbers is a non-empty list) and B3 (where the variable x is not equal to 50), are extracted as targets for generating supplementary test cases.

[0050] S2064: Generate test cases to be added based on the uncovered boundary domains.

[0051] In this step, for each uncovered boundary domain, test cases can be generated according to the corresponding rules based on its domain type (discrete / continuous / mixed) to ensure that the newly generated test cases can accurately cover the constraints of the boundary domain.

[0052] For example, if the uncovered boundary domain B3 is a discrete domain (variable x is an int, condition is not equal to 50), then test case "x=50" is generated to cover this boundary; if the uncovered boundary domain is a continuous domain (variable x is an int, lower bound 0 inclusive, upper bound 100 inclusive), then three test cases "x=0", "x=100", and "x=50" are generated; if it is a mixed domain (variable x is an int, lower bound 0 inclusive and not equal to 10), then three test cases "x=0", "x=10", and "x=20" are generated, respectively covering the boundaries of the continuous and discrete parts.

[0053] S2065: Combine valid test cases and test cases to be added to obtain optimized test cases.

[0054] In this step, the remaining valid test cases after deduplication are merged with the newly generated test cases to be added to form a complete set of optimized test cases, ensuring no redundancy and coverage of all boundary domains.

[0055] As described in the above embodiments, this disclosure first selects valid test cases that meet the correctness standards based on the test coverage analysis report, and eliminates invalid test cases to ensure the basic reliability of the test cases. Then, it performs redundancy removal on the valid test cases, eliminating test cases that repeatedly cover the same boundary and are undifferentiated, thereby achieving streamlined and efficient test cases. It locates the uncovered boundary domains in the report to clarify the test coverage gaps. Based on the type of the uncovered boundary domains, it generates supplementary test cases according to the corresponding rules to fill the coverage gaps. This series of operations synergistically achieves the "purification and optimization" of existing test cases and the "gap filling" of boundary coverage, resulting in optimized test cases that are complete in boundary coverage, without redundancy, and effective.

[0056] In one possible implementation, the test coverage analysis report includes the correctness attributes of the test cases to be optimized.

[0057] In step S2061 above, based on the test coverage analysis report, the effective test cases in the test cases to be optimized are determined, including:

[0058] S611: Determine the test cases to be optimized in the test coverage analysis report whose correctness attribute is the target value as valid test cases.

[0059] In this step, a target value for the correctness attribute is preset (such as "True" or "1"). The test case attributes in the analysis report are traversed, and the test cases to be optimized whose attribute values ​​match the target value are directly marked as valid test cases.

[0060] As can be seen from the description of the above embodiments, the embodiments of this disclosure determine whether the test cases to be optimized are valid test cases by reading the correctness attributes in the test coverage analysis report, thereby achieving accurate positioning of valid test cases.

[0061] In one possible implementation, step S2064 above generates test cases to be added based on the uncovered boundary domain, including:

[0062] S641: If the uncovered boundary domain is a discrete domain, then generate test cases to be added based on the values ​​of the uncovered boundary domain.

[0063] In this step, the discrete domain consists of multiple isolated legal or illegal values. For the boundary of the discrete domain, all isolated values ​​within the domain are extracted, and corresponding test cases are generated for each value to ensure coverage of all legal and illegal values ​​in the domain.

[0064] For example, if the discrete domain is not covered as "variable status (integer type), with values ​​of 0, 1, and 2", then these three isolated values ​​are extracted, and test cases "status=0", "status=1", and "status=2" are generated respectively to fully cover all legal values ​​in the discrete domain; if the discrete domain condition is "variable type (string type), not equal to 'normal'", then test case "type='normal'" is generated to specifically cover the boundary of this illegal value.

[0065] S642: If the uncovered boundary domain is a continuous domain, determine the endpoint values ​​and at least one intermediate value corresponding to the uncovered boundary domain. Generate test cases to be added based on the endpoint values ​​and intermediate values.

[0066] In this step, a continuous domain corresponds to a continuous range of values. For the boundaries of a continuous domain, first identify the upper and lower endpoint values ​​of the range (including whether it contains endpoints), then select 1-2 typical intermediate values ​​within the range, and generate test cases based on the endpoint values ​​and the intermediate values ​​respectively.

[0067] For example, if the continuous field not covered is "variable score (integer type), lower bound 60 (inclusive), upper bound 100 (inclusive)," then first determine the endpoint values ​​60 and 100, then select the intermediate value 80 to generate test cases "score=60", "score=100", and "score=80". This covers both the endpoints of the interval and verifies the logic within the interval using the intermediate value. More test cases such as "score=75" and "score=66" can also be generated.

[0068] S643: If the uncovered boundary domain is a mixed domain, determine the endpoint values ​​and at least one intermediate value corresponding to the uncovered boundary domain. Based on the endpoint values, intermediate values, and the values ​​of the uncovered boundary domain, generate test cases to be added.

[0069] In this step, the hybrid domain contains both discrete and continuous components. For the boundary of the hybrid domain, the continuous and discrete components are processed separately. Test cases are generated for the continuous component based on endpoint values ​​plus intermediate values, and test cases are generated for the discrete component based on isolated values. These are then integrated to form the test cases to be added that cover the hybrid domain.

[0070] For example, if the mixed domain is not covered as "variable age (integer type), continuous domain 18 (inclusive) to 60 (inclusive), discrete domain not equal to 30", then the continuous part will generate "age=18", "age=60", and "age=40", and the discrete part will generate "age=30", which will be integrated into four groups of test cases to be added, covering both the continuous interval and the discrete constraint of the mixed domain.

[0071] As can be seen from the description of the above embodiments, the embodiments of this disclosure supplement the missing test cases by generating corresponding test cases for discrete domain, continuous domain and mixed domain respectively.

[0072] In one possible implementation, step S2062 above involves removing redundant use cases from the valid use cases to obtain the remaining use cases, including:

[0073] S62A1: Determine the test cases at non-boundary endpoints in the valid test cases corresponding to the target boundary domain as non-boundary test cases.

[0074] In this step, the target boundary domain can be a continuous domain, and the test cases for non-boundary endpoints can be test cases for the non-boundary endpoints of the input values.

[0075] For example, a continuous domain such as "[3, 90]" has boundary endpoints of 3 and 90, while non-boundary test cases include 5, 10, 15, 20, etc.

[0076] S62A2: If the number of non-boundary test cases is greater than the preset number, calculate the difference between the preset number and the number of test cases for non-boundary endpoints.

[0077] In this step, the preset number can be a value set by the staff based on historical test experiments or test results.

[0078] The preset number of items is, for example, 2, 3, 4, etc.

[0079] For example, with 10 non-boundary test cases and a preset number of 3, the calculated difference is 7.

[0080] S62A3: Identify non-boundary test cases with a difference in number as redundant test cases.

[0081] In this step, for example, if the difference is 5, then 5 non-boundary test cases are randomly selected as redundant test cases. Or, for example, if the difference is 2, then 2 non-boundary test cases are randomly selected as redundant test cases.

[0082] S62A4: Remove redundant test cases to obtain the remaining test cases.

[0083] In this step, redundant test cases can be deleted to obtain the remaining test cases.

[0084] As can be seen from the description of the above embodiments, the embodiments of this disclosure reduce the number of redundant test cases, test duration, and computational resource waste by counting the number of non-boundary test cases and removing some redundant non-boundary test cases.

[0085] In one possible implementation, S202 performs static analysis on the code segment to be tested to obtain the boundary assertions corresponding to the code segment to be tested, including:

[0086] S2021: Determine the abstract syntax tree of the code snippet to be tested.

[0087] In this step, a code parser can be used to break down the code snippets into their syntax, generating an abstract syntax tree that reflects the code's structure and logical relationships, thus clarifying the hierarchy and connections of each syntax unit.

[0088] For example, if we select a simple function that takes two integer parameters, performs an addition operation, and returns an integer result, the code parser will break it down into the corresponding target syntax nodes such as function definition, parameters (including type declarations), return statement, and addition operation. Each node forms a hierarchical structure according to the function syntax, representing the composition and logical relationship of the function.

[0089] S2022: Identify the target syntax node in the abstract syntax tree.

[0090] In this step, the abstract syntax tree can be traversed to filter out nodes of types such as comparison operations, Boolean operations, conditional loop test expressions, assertion statements, and annotated function parameters. These nodes are the core carriers of boundary conditions.

[0091] S2023: Based on the preset boundary domain keyword library, map the boundary conditions in the target syntax node to boundary assertions, wherein the boundary domain keyword library includes the correspondence between boundary conditions and boundary assertions.

[0092] In this step, based on the boundary conditions, the correspondence between boundary conditions and boundary assertions in the boundary domain keyword library is searched to obtain boundary assertions in a unified format.

[0093] The preset boundary domain keyword library can be pre-defined by staff and stored in formats such as tables, key-value pairs, or files. The boundary domain keyword library can contain keywords such as lower bound, upper bound, empty, not empty, zero, non-zero, contain, not equal to, and not contained in. As described in the above embodiments, this disclosure, by determining the abstract syntax tree of the code segment to be tested and using the boundary domain keyword library to map the boundary conditions in the abstract syntax tree to boundary assertions, completes the transformation from code structure to explicit and unified boundary rules. This provides accurate and usable core input for the subsequent generation of a structured boundary list, achieving automated extraction and standardized representation of deep boundary conditions in the code.

[0094] In one possible implementation, step S2022 above, identifying the target syntax node in the abstract syntax tree, includes:

[0095] S20221: Identify the syntax nodes in the abstract syntax tree that contain preset fields as target syntax nodes.

[0096] In this step, the syntax nodes in the abstract syntax tree that contain comparison operators, Boolean operators, test conditions, assertions, and function parameters can be identified as target syntax nodes.

[0097] The comparison operators include "==", "!=", "<", "<=", ">", ">=", "in", "notin", "is", and "isnot", while Boolean operators include "and" and "or". Test conditions can be those found in conditional statements and loops, such as "if", "for", and "while", and assertions such as "assert". Function parameters can include parameters with default values ​​or type annotations.

[0098] As can be seen from the description of the above embodiments, the embodiments of this disclosure determine the syntax node containing the preset field as the target syntax node, thereby extracting a specific syntax node from the abstract syntax tree, which is convenient for subsequent conversion into boundary assertions.

[0099] In one possible implementation, step S205 above involves inputting structured prompt words into a pre-trained language model to obtain at least one test coverage analysis report, including step S2051 or step S2052.

[0100] S2051: Input the structured prompt words into at least one pre-trained language model to obtain at least one test coverage analysis report.

[0101] In this step, multiple different pre-trained language models are selected, and the same structured prompt word is input into different pre-trained language models. Test coverage analysis reports output by each model are obtained independently, providing multi-source results for subsequent cross-validation.

[0102] S2052: Input the structured prompt words into the pre-trained language model at least once to obtain at least one test coverage analysis report.

[0103] In this step, for a single pre-trained language model, the structured prompt words are repeatedly input multiple times, and the test coverage analysis report of each output is recorded. The impact of model randomness is reduced by comparing multiple results.

[0104] As can be seen from the description of the above embodiments, the embodiments of this disclosure can obtain multiple reports through independent reasoning of multiple models, or obtain results by calling a single model multiple times, which can form a cross-validation basis for comparing multiple results, effectively filter unreliable outputs caused by the randomness of pre-trained language model reasoning, dynamically optimize the reasoning process, improve the accuracy, stability and robustness of test coverage analysis reports, and provide a basis for subsequent test case optimization work based on reports.

[0105] In one possible implementation, after obtaining 5 test coverage analysis reports, if the correctness of variable A is "yes" in 4 test coverage analysis reports and "no" in the other test coverage analysis report, then "yes" is used as the test coverage analysis report to be used. As another example, after obtaining 7 test coverage analysis reports, if the name of the uncovered variable is "variable B" in 5 test coverage analysis reports and "variable C" in the other 2 test coverage analysis reports, then "variable B" is used as the name of the uncovered variable.

[0106] In one possible implementation, step S204 above involves determining structured prompt words based on the structured boundary list, the code snippet to be tested, and the test cases to be optimized, including:

[0107] S2041: Combine the preset role setting fields, preset task setting fields, preset output specification fields, structured boundary list, code snippets to be tested, and test cases to be optimized to obtain structured prompt words.

[0108] In this step, the fields are assembled in a fixed order. The role setting field defines the analysis identity of the model, the task setting field clarifies the core objectives of the test coverage analysis, the output specification field limits the report format, and then the structured boundary list, the code snippets to be tested, and the test cases to be optimized are combined to form a complete structured prompt word.

[0109] The preset role setting fields include phrases such as "You are a rigorous software analysis and testing expert" and "Answer as a senior software analysis and testing expert." Preset task setting fields include phrases such as "Analyze whether the given unit tests fully cover all boundary conditions involved in the target code" and "Analyze whether the following test cases fully cover all boundary conditions involved in the target code." Preset output specification fields include phrases such as "Output the test case name, whether the test case exists in the test set, the name of the boundary domain to which it belongs, and finally output the percentage of boundary domains with test cases out of the total number of boundary domains, and the names of boundary domains without test cases."

[0110] As can be seen from the description of the above embodiments, the embodiments of this disclosure obtain structured prompt words by combining role setting fields, preset task setting fields, preset output specification fields, structured boundary lists, code snippets to be tested, and test cases to be optimized, thereby realizing standardized prompt word construction. This enables the pre-trained language model to output a standard test coverage analysis report based on the prompt words, providing a foundation for subsequent optimization of test cases.

[0111] In one possible implementation, in step S202 above, static analysis is performed on the code segment to be tested to obtain the boundary assertions corresponding to the code segment to be tested.

[0112] For example, the code snippet to be tested calculates the arithmetic mean of target numbers. The calculation process involves calculating the absolute deviation of each number "number" from the mean, summing these deviations, and then averaging them (i.e., the mean absolute deviation). An abstract syntax tree is constructed from the code under test, and static analysis is performed. This results in "number" being a list with float elements. The list cannot be empty; otherwise, a division by zero exception will be triggered. Therefore, exception handling needs to be tested. Based on a predefined boundary domain keyword library, the requirement that the list cannot be empty is mapped to a boundary assertion.

[0113] Accordingly, in step S203, a structured boundary list is determined based on the boundary assertions, including: using "number" as the variable name, "list" as the variable type, defining "cannot be empty" as a boundary assertion, defining discrete as the boundary domain type, and defining "yes" as the value of whether the variable contains the boundary value attribute and whether the variable is required. The obtained structured boundary list is then used to replace the target fields in the template (a combination of preset role setting fields, preset task setting fields, preset output specification fields, the structured boundary list, the code snippet to be tested, and the test cases to be optimized) to obtain structured prompts.

[0114] The structured prompts are input into the pre-trained language model to obtain a test coverage analysis report. If the test coverage analysis report shows no covered boundary regions, discrete values ​​are generated to supplement the test cases.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0116] Figure 3 This is a schematic diagram of the test case generation device provided in an embodiment of this application. Figure 3 As shown, embodiments of this application also provide a test case generation device 300. The test case generation device 300 includes: a data acquisition module 301, a static analysis module 302, a list determination module 303, a prompt word determination module 304, a report acquisition module 305, and a test case generation module 306.

[0117] The data acquisition module 301 is used to acquire the test cases to be optimized and the code snippets to be tested.

[0118] The static analysis module 302 is used to perform static analysis on the code segment to be tested and obtain the boundary assertions corresponding to the code segment to be tested.

[0119] The list determination module 303 is used to determine a list of structured boundaries based on boundary assertions.

[0120] The prompt word determination module 304 is used to determine structured prompt words based on the structured boundary list, the code snippet to be tested, and the test cases to be optimized.

[0121] The report acquisition module 305 is used to input structured prompt words into a pre-trained language model and obtain at least one test coverage analysis report.

[0122] The test case generation module 306 is used to generate optimized test cases based on the test coverage analysis report and the test cases to be optimized.

[0123] For a description of the features in the embodiment corresponding to the test case generation device, please refer to the relevant description in the embodiment corresponding to the test case generation method, which will not be repeated here.

[0124] In one possible implementation, the test case generation module 306 is used to determine the valid test cases in the test cases to be optimized based on the test coverage analysis report; remove redundant test cases from the valid test cases to obtain the remaining test cases; read the uncovered boundary domains in the test coverage analysis report; generate test cases to be added based on the uncovered boundary domains; and combine the valid test cases and the test cases to be added to obtain the optimized test cases.

[0125] In one possible implementation, the test coverage analysis report includes the correctness attributes corresponding to the test cases to be optimized; the test case generation module 306 is used to determine the test cases to be optimized with the correctness attributes of the test coverage analysis report as target values ​​as valid test cases.

[0126] In one possible implementation, the test case generation module 306 is used to generate test cases to be added based on the values ​​of the uncovered boundary domain if the uncovered boundary domain is a discrete domain; if the uncovered boundary domain is a continuous domain, determine the endpoint values ​​and at least one intermediate value corresponding to the uncovered boundary domain; and generate test cases to be added based on the endpoint values ​​and intermediate values; if the uncovered boundary domain is a mixed domain, determine the endpoint values ​​and at least one intermediate value corresponding to the uncovered boundary domain; and generate test cases to be added based on the endpoint values, intermediate values, and values ​​of the uncovered boundary domain.

[0127] In one possible implementation, the test case generation module 306 is used to identify test cases at non-boundary endpoints among the valid test cases corresponding to the target boundary domain as non-boundary test cases; if the number of non-boundary test cases is greater than a preset number, the difference between the preset number and the number of test cases at non-boundary endpoints is calculated; the non-boundary test cases whose number is equal to the difference are identified as redundant test cases; and the redundant test cases are removed to obtain the remaining test cases.

[0128] In one possible implementation, the static analysis module 302 is used to determine the abstract syntax tree of the code segment to be tested; identify the target syntax node in the abstract syntax tree; and map the boundary conditions in the target syntax node to boundary assertions according to a preset boundary domain keyword library, wherein the boundary domain keyword library includes the correspondence between boundary conditions and boundary assertions.

[0129] In one possible implementation, identifying the target syntax node in the abstract syntax tree includes: determining the syntax node in the abstract syntax tree that contains a preset field as the target syntax node.

[0130] In one possible implementation, the report acquisition module 305 is used to input structured prompts into at least one pre-trained language model to obtain at least one test coverage analysis report; or, to input structured prompts into the pre-trained language model at least once to obtain at least one test coverage analysis report.

[0131] In one possible implementation, the prompt word determination module 304 is used to combine preset role setting fields, preset task setting fields, preset output specification fields, structured boundary list, code snippets to be tested, and test cases to be optimized to obtain structured prompt words.

[0132] For a description of the features in the embodiment corresponding to the test case generation device, please refer to the relevant description in the embodiment corresponding to the test case generation method, which will not be repeated here.

[0133] Figure 4 A schematic diagram of the structure of the electronic device provided in this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the electronic device 40 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus.

[0134] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to execute the above-described test case generation method embodiment.

[0135] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0136] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0137] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0138] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0139] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described test case generation method embodiments at runtime.

[0140] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0141] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described test case generation method embodiments.

[0143] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described test case generation method embodiments.

[0144] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0145] The foregoing has provided a detailed description of a test case generation method, device, storage medium, and program product provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to aid in understanding the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A test case generation method, characterized in that, include: Obtain the test cases to be optimized and the code snippets to be tested; Static analysis is performed on the code segment to be tested to obtain the boundary assertions corresponding to the code segment to be tested. Based on the boundary assertions, determine the list of structured boundaries; Based on the structured boundary list, the code snippet to be tested, and the test cases to be optimized, determine the structured prompt words; Input the structured prompt words into the pre-trained language model to obtain at least one test coverage analysis report; Based on the test coverage analysis report and the test cases to be optimized, optimized test cases are generated; The step of performing static analysis on the code segment to be tested to obtain the boundary assertions corresponding to the code segment to be tested includes: determining the abstract syntax tree of the code segment to be tested; identifying the target syntax node in the abstract syntax tree; and mapping the boundary conditions in the target syntax node to the boundary assertions according to a preset boundary domain keyword library, wherein the boundary domain keyword library includes the correspondence between boundary conditions and boundary assertions. The step of determining the structured boundary list based on the boundary assertions includes: classifying and organizing the boundary assertions according to a preset data format specification, aggregating them by variable dimension, and supplementing variable types, boundary types, or value constraints to form a structured boundary list.

2. The method according to claim 1, characterized in that, The step of generating optimized test cases based on the test coverage analysis report and the test cases to be optimized includes: Based on the test coverage analysis report, the effective test cases in the test cases to be optimized are identified; Remove redundant use cases from the valid use cases to obtain the remaining use cases; Read the uncovered boundary regions from the test coverage analysis report; Based on the uncovered boundary regions, generate test cases to be added; The effective test cases and the test cases to be added are combined to obtain the optimized test cases.

3. The method according to claim 2, characterized in that, The test coverage analysis report includes the correctness attributes of the test cases to be optimized. The step of determining the valid test cases among the test cases to be optimized based on the test coverage analysis report includes: Test cases whose correctness attribute is the target value in the test coverage analysis report are identified as valid test cases.

4. The method according to claim 2, characterized in that, The step of generating test cases to be added based on the uncovered boundary region includes: If the uncovered boundary domain is a discrete domain, then the test cases to be added are generated based on the values ​​of the uncovered boundary domain. If the uncovered boundary domain is a continuous domain, then determine the endpoint value and at least one intermediate value corresponding to the uncovered boundary domain; generate the test cases to be added based on the endpoint value and the intermediate value. If the uncovered boundary domain is a mixed domain, then determine the endpoint value and at least one intermediate value corresponding to the uncovered boundary domain; generate the test cases to be added based on the endpoint value, the intermediate value and the value of the uncovered boundary domain.

5. The method according to claim 2, characterized in that, The process of removing redundant use cases from the valid use cases to obtain the remaining use cases includes: Among the valid test cases corresponding to the target boundary domain, the test cases at non-boundary endpoints are identified as non-boundary test cases; If the number of non-boundary test cases is greater than the preset number, then calculate the difference between the preset number and the number of test cases for the non-boundary endpoints; The number of non-boundary test cases equal to the difference is determined as redundant test cases; Remove the redundant use cases to obtain the remaining use cases.

6. The method according to claim 1, characterized in that, The process of identifying the target syntax node in the abstract syntax tree includes: The syntax nodes containing preset fields in the abstract syntax tree are identified as the target syntax nodes.

7. The method according to any one of claims 1 to 5, characterized in that, The step of inputting the structured prompt words into a pre-trained language model to obtain at least one test coverage analysis report includes: Input the structured prompts into at least one pre-trained language model to obtain at least one test coverage analysis report; or, The structured prompt words are input into the pre-trained language model at least once to obtain at least one test coverage analysis report.

8. The method according to any one of claims 1 to 5, characterized in that, The step of determining structured prompt words based on the structured boundary list, the code snippet to be tested, and the test cases to be optimized includes: The structured prompt words are obtained by combining the preset role setting fields, preset task setting fields, preset output specification fields, the structured boundary list, the code snippet to be tested, and the test cases to be optimized.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as described in any one of claims 1 to 8.

Images