Test instance generation method, device and equipment

Abstract

A test case generation method, apparatus and device, the method comprising: obtaining test data of a to-be-tested module included in a to-be-tested engineering project, the test data comprising a program code of the to-be-tested module and call analysis information, the calling analysis information is used for indicating a calling relationship between the to-be-tested module and other functional modules included in the engineering project; constructing first prompt information based on the test data, and inputting the first prompt information into a pre-training language model to obtain a first test case of the to-be-tested module generated by the pre-training model, the first prompt information is used for prompting the pre-training language model to construct test cases for calling different program code branches in the to-be-tested module by different function modules based on calling analysis information.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a method, apparatus, and device for generating test cases. Background Technology

[0002] Existing software testing technologies encompass both manual and automated testing. Manual testing relies on testers to verify functionality step-by-step; while flexible, it is inefficient and susceptible to human error. Automated testing, through scripts and tools, improves testing speed and coverage while reducing human error. Automated testing techniques include unit testing, integration testing, functional testing, and regression testing, and are widely used in continuous integration / continuous deployment (CI / CD) pipelines, significantly improving software development efficiency and quality.

[0003] Existing test case generation technologies fall into two main categories: traditional methods and methods based on large models.

[0004] Traditional methods, such as Pynguin, are advanced search-based test generation (SBST) techniques used for dynamically typed programming languages ​​like Python. They initiate the testing process by taking the Python code to be tested as input and employ one of the search-based algorithms (including MIO, MOSA, and DynaMOSA) to generate new test cases by changing the values ​​and statements included in the seed. The seed is a closely approximation of manually written test cases, used to overcome the shortcomings of random test case generation methods, ensuring the validity and correctness of the test cases generated by Pynguin based on the seed. However, after a period of searching, Pynguin encounters a test coverage bottleneck. This is because Pynguin generates test cases randomly. As Pynguin evolves the input data, it struggles to randomly generate test cases that satisfy complex constraints or conform to a specific format, thus resulting in low test coverage.

[0005] Large-model-based methods, such as CODAMOSA, are advanced test generation tools based on large language models (LLMs) designed to improve test case generation techniques for SBST. CODAMOSA's main contribution is helping SBST escape local optima by generating seeds for Pynguin through prompts in the LLM. More specifically, CODAMOSA can be roughly divided into four steps: when Pynguin gets stuck in a local optimum, CODAMOSA switches to calling the LLM, generating new seeds by incorporating as much of the source code of the module under test as possible into the prompts, and breaking down the tests into a format that allows Pynguin to continue evolving. Then, it switches back to Pynguin and runs on these new seeds until a given test budget is reached.

[0006] It can be seen that existing methods based on large models still rely on traditional methods to generate test cases, which is still not enough to solve the shortcomings of traditional methods. Summary of the Invention

[0007] This application provides a test case generation method, apparatus, and device, which provides a test case generation method that does not rely on traditional methods, breaks through the coverage bottleneck, and improves test coverage.

[0008] In a first aspect, this application provides a test case generation method, which includes: acquiring test data of a module to be tested included in an engineering project to be tested, the test data including program code and call analysis information of the module to be tested, wherein the call analysis information is used to indicate the call relationship between the module to be tested and other functional modules included in the engineering project; then, constructing prompt information (denoted as first prompt information) based on the acquired test data, for example, the first prompt information including the test data, used to instruct a pre-trained language model to construct test cases in which different functional modules call different branches of program code in the module to be tested; inputting the first prompt information into the pre-trained language model to obtain a first test case of the module to be tested generated by the pre-trained model, the first test case being used to test the module to be tested, the obtained test result indicating whether the function of the program code branch covered by the first test case is correct, or the first test case being sent to a user for testing by the user.

[0009] Through the above design, prompts for the pre-trained language model are constructed based on the program code and call analysis information of the module under test. The call analysis information helps the pre-trained language model explore different entry paths based on the call relationships between the functions under test. This not only helps to construct objects involved in effective program code, but also helps to capture various use cases where specific attribute values ​​are required for program code remainders that may arise in practice. This enables the pre-trained language model to generate test cases that can cover program code branches in the module under test that are difficult to cover, thereby enhancing the diversity and coverage of test cases.

[0010] In one alternative design, the call analysis information includes a forward call sequence and / or a backward call sequence of the module under test; wherein the forward call sequence indicates one or more other functional modules called by the module under test, and the backward call sequence indicates the call path of one or more other functional modules calling the module under test.

[0011] Through the above design, backward call analysis is used to extract the calling patterns of the function under test within the project, thereby revealing its usage and addressing the challenge of constructing complex objects. Forward call analysis extracts information about the external functions that the function under test depends on, helping the pre-trained language model better understand the function's functionality, especially its complex branching logic. This allows for the generation of code branches that traditional methods struggle to cover, improving test coverage.

[0012] In an optional design, the test data also includes test coverage information of the module under test. The test coverage information is used to indicate the program code branches in the module under test that are covered by at least one second test case. The second test case is a test case generated before the first test case. The first test case is used to test the program code branches in the module under test that are not covered.

[0013] In one alternative design, the test coverage information includes at least one second test case.

[0014] Through the above design, existing test cases can be used as examples to guide the pre-trained language model to generate test cases for program code branches that are not yet covered, thereby improving the overall testing efficiency. Furthermore, the pre-trained language model can learn from the examples to generate test cases, thereby improving the legality and accuracy of the test cases generated by LLM.

[0015] In an optional design, the method further includes: updating test coverage information based on the first test case to obtain updated test coverage information, wherein the updated test coverage information is used to indicate program code branches in the module under test that are covered by the first test case and at least one second test case; inputting second prompt information into a pre-trained language model to obtain a third test case generated by the pre-trained language model; wherein the second prompt information includes the updated test coverage information, and the third test case is used to test the uncovered program code branches in the module under test.

[0016] In one alternative design, acquiring test data for the modules under test included in the project to be tested includes: providing an application programming interface (API) to the user, the API indicating multiple fields representing different data items included in the test data; and receiving test data sent by the user, the test data including multiple fields and data content entered by the user for each field.

[0017] In one alternative design, acquiring test data for the modules to be tested included in the project to be tested includes: providing a configuration interface to the user; receiving test data input or selected by the user in the configuration interface, wherein the test data includes multiple data configuration items and data content input or selected by the user for each data configuration item.

[0018] In an alternative design, the method is applied to a cloud management platform; obtaining test data of the modules to be tested included in the project to be tested, including: the cloud management platform determining the test data input or selected by the user on the cloud management platform; the method further includes: the cloud management platform sending a set of test cases to the user, the set of test cases including a second set of test cases.

[0019] Secondly, this application also provides a computing device that has the functionality to implement the behavior in the method example of the first aspect described above. The beneficial effects are described in the first aspect description and will not be repeated here. The functionality can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functionality. In one possible design, the device structure includes an acquisition module and a processing module. In another possible design, the acquisition module and the processing module may be the same module. These modules can perform the functionality of the behavior in the method example of the first aspect described above; see the detailed description in the method example for details, which will not be repeated here.

[0020] Thirdly, this application also provides a computing device cluster, which includes at least one computing device. This at least one computing device has the functionality to implement the behavior described in the method example of the first aspect above. The beneficial effects are described in the first aspect and will not be repeated here. Each computing device includes a processor and a memory. The processor is configured to support the computing device in executing the method described in the first aspect or any possible design of the first aspect. The memory is coupled to the processor and stores the necessary program instructions and data of the computing device. The computing device also includes a communication interface for communicating with other devices.

[0021] Fourthly, this application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or any possible design of the first aspect.

[0022] Fifthly, this application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in the first aspect or any possible design of the first aspect.

[0023] Sixthly, this application also provides a computer chip connected to a memory, the chip being used to read and execute a software program stored in the memory, and to execute the method described in the first aspect or any possible design of the first aspect.

[0024] For the beneficial effects of aspects two through six, please refer to the beneficial effects of aspect one, which will not be repeated here. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the structure of a software engineering project provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of the architecture of a cloud service system provided in an embodiment of this application;

[0027] Figure 3 This is a flowchart illustrating a test case generation method provided in an embodiment of this application.

[0028] Figure 4 This is a schematic diagram of a configuration interface provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram illustrating the construction process of a backward call sequence provided in an embodiment of this application;

[0030] Figure 6 This is a schematic diagram illustrating the construction process of a forward call sequence provided in an embodiment of this application;

[0031] Figure 7 This is a schematic diagram of a test case generation process based on an IDE plugin provided in an embodiment of this application;

[0032] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0033] Figure 9 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0034] Figure 10 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0035] Figure 11 This is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0036] Figure 1 This is a schematic diagram of a software testing scenario provided in an embodiment of this application, such as... Figure 1 As shown, the software comprises multiple modules, each consisting of one or more functions. Each function implements a specific function through a piece of code. Software testing refers to using given test cases to pre-run the code of each module to test whether the module can achieve the intended function; simply put, it's testing whether the code is accurate.

[0037] In practical applications, a module typically contains a large amount of code and complex logic, requiring a significant number of test cases for testing. Each module may contain several branches of code, such as loops and conditional statements, and each test case can be used to test (cover) a portion of those branches. Within a given testing timeframe, higher test coverage is better, as this reduces the probability of users encountering bugs after the software is released.

[0038] However, existing solutions suffer from a test coverage bottleneck in generating test cases. That is, after a certain period, the test coverage of the generated test cases will no longer improve.

[0039] In view of this, embodiments of this application provide a test case generation method. In this method, call analysis information of the module under test in the software engineering project to be tested is constructed to indicate the call relationship between the module under test and other functional modules in the project. Based on the program code of the module under test and the call analysis information, the prompt information of the pre-trained language model (LLM) is enhanced. This prompt information helps the LLM explore different entry paths, which not only helps to construct objects involved in effective program code, but also helps to capture various use cases that may arise in practice and require specific attribute values ​​for program code remainders. This enables the LLM to generate test cases that can cover program code branches in the module under test that are difficult to cover, thereby enhancing the diversity and coverage of test cases.

[0040] Figure 2 An exemplary schematic diagram of a cloud service system is shown.

[0041] like Figure 2 As shown, the cloud service system 100 includes a cloud service module 110 and a cloud management platform 120 provided by a cloud vendor. Optionally, the cloud service system 100 also includes a back-end storage device 130.

[0042] The cloud management platform 120 can connect to the terminal devices operated by tenants via the Internet. Tenants can subscribe to cloud services from the cloud provider, so that the cloud service module 110 can provide corresponding services to the tenants according to the cloud services they have subscribed to. Within the system 100, the cloud management platform 120 can connect to the cloud service module 110 and the back-end storage 130 via the internal network.

[0043] Tenants can purchase cloud services on the cloud management platform 120. After successful purchase, the cloud management platform 120 notifies the cloud service module 110 to provide services to the tenant. For example, the cloud service module 110 can provide a test generation service, which generates test cases for software engineering projects. After the tenant successfully pays, the cloud management platform 120 notifies the cloud service module 110 to generate test cases for the software specified by the tenant and returns the generated test cases to the tenant. The backend storage 130 can be used to store relevant data of the tenant, such as other data of the tenant, such as data of the software engineering project to be tested, such as program code. For another example, it can be used to store the test cases generated by the cloud service module 110 for the tenant. When the test coverage meets the requirements, the cloud management platform 120 retrieves multiple test cases for the tenant from the backend storage 130 and returns them to the tenant.

[0044] The back-end storage 130 may include at least two types of back-end storage modules, such as a memory-type back-end storage module and a persistent back-end storage module. The persistent back-end storage module may include, for example, an SSD-type back-end storage module of the type of solid state disk (SSD) or an OBS back-end storage module of the type of object storage service (OBS) bucket.

[0045] It should be noted that, Figure 2 The system 100 structure shown is merely an example; the systems or devices that may be applicable to the embodiments of this application may include relative... Figure 2 The system may have more or fewer components; for example, in another implementation, system 100 does not include back-end memory 130. This application embodiment does not limit this.

[0046] The following is applied to Figure 2 Taking the system 100 shown as an example, the test case generation method provided in this application embodiment will be described in detail.

[0047] Figure 3 This is a flowchart illustrating a test case generation method provided in an embodiment of this application. Figure 3 As shown, the method may include the following steps:

[0048] Step 301: Obtain the input data entered or selected by the tenant on the cloud management platform.

[0049] This application supports tenants inputting various types of input data. For example, input data may include the complete program code of the software engineering project to be tested. Alternatively, input data may include the program code of the functional module to be tested (hereinafter referred to as the module under test) within the project. Alternatively, input data may include the program code of the module under test and its call analysis information. Alternatively, input data may include the program code of the module under test, call analysis information, and test coverage information.

[0050] There are multiple ways for the cloud management platform 120 to obtain input data provided by the tenant. In one implementation, the cloud management platform 120 can provide an API interface for the tenant to call, and the tenant can send input data by calling the API interface through the user terminal.

[0051] Specifically, the API interface is used to indicate multiple fields representing different data items included in the input data. These multiple fields include, but are not limited to, one or more fields representing the program code of the module under test, the call analysis information of the module under test, and the test coverage information of the module under test.

[0052] As an example, the API format provided by the cloud management platform 120 is as follows:

[0053]

[0054] Specifically, the cloud management platform 120 can display the above API format on a webpage provided on the internet, and indicate the usage of the corresponding fields, such as the relevant prompts after / / above. After seeing the above API format, the tenant fills in the corresponding data according to the above API format, for example, filling in "TGS (Test generation service)" after "namespace":, that is, "namespace":"TGS", indicating that the service involved is the test generation service purchased by the tenant.

[0055] For example, tenants can fill in the data in the API format as follows:

[0056]

[0057] Tenants can send the API with the parameters input above to the cloud management platform 120 via the Internet in the form of a template. The cloud management platform 120 detects the data corresponding to different fields in the API, such as file identifiers (such as file name or file path and other identifiers). The file uniquely indicated by these file names can be stored in the back-end storage 130. After obtaining the file identifiers corresponding to different fields in the API, the cloud management platform 120 retrieves the corresponding files from the back-end storage 130.

[0058] As another example, tenants can call the API to transfer the program code of the module under test, call analysis information, test coverage information, and other data files.

[0059] In another embodiment, in addition to providing an API, such as Figure 4 As shown, the cloud management platform 220 can also provide a console interface for tenants to configure. The tenant's terminal device can display this console interface, where the tenant can input or select data similar to the API described above in the relevant configuration items.

[0060] It should be noted that the API format and console interface described above both display the call analysis information and test coverage information of the module under test. In another design, these two items may not be included in the above two embodiments. Alternatively, in a possible scenario, the user may not have configured these two items, in which case the input data obtained by the cloud management platform 120 may not include the call analysis information and / or test coverage information of the module under test. When the input data does not include call analysis information, the cloud management platform 120 notifies the cloud service module 110 to generate the call analysis information of the module under test (see step 302). And / or, the cloud management platform 120 notifies the cloud service module 110 to determine the test coverage information of the module under test (see step 303).

[0061] Step 302: Generate call analysis information for the module under test.

[0062] The technical personnel who analyzed this application identified the following specific reasons for the defects in existing LLM applications: some program code branch constraints involve objects with complex construction processes. Specifically, the construction of certain objects depends on other (complex) types of objects, and the compilation success rate of test cases generated by existing LLMs is relatively low (approximately 39%), highlighting the inadequacy of LLMs in this regard. Therefore, existing LLM-based test generation technologies (such as CODAMOSA) cannot generate valid objects for these scenarios, and the accuracy of existing LLM-generated test cases is even lower for program code branch constraints that require objects with specific attribute values.

[0063] Secondly, some program code branch constraints involve complex inter-procedural dependencies. In such cases, existing LLM techniques, based on the source code of the input target function or some coarse-grained contextual information, are insufficient for the LLM to understand the semantics conveyed by a series of called functions, making it difficult to generate effective test cases.

[0064] To address this issue, this application proposes constructing call analysis information for the module under test, which will then be used to prompt the LLM to generate test cases.

[0065] Call analysis information is used to indicate the call relationships between the module under test and other functional modules included in the aforementioned software engineering project. For example, call analysis information includes a call sequence containing the module under test.

[0066] First, we introduce the call sequence, which indicates the call path between functional modules. For example, a call sequence includes multiple functional modules arranged in the order of their calls. One call sequence includes: Module 1, Module 2, and Module 3, indicating that Module 1 calls Module 2, and Module 2 calls Module 3. Each functional module may include one or more functions. In this embodiment, the call sequence can be constructed at the function level, in which case the call sequence includes multiple functions arranged in the order of their calls.

[0067] Based on the position of the module under test in the call sequence, the call sequence can be divided into backward call sequences and forward call sequences. A backward call sequence refers to the call sequence that ends with the module under test, while a forward call sequence refers to the call sequence that starts with the module under test.

[0068] Example 1: The call analysis information includes the backward call sequence of the module under test.

[0069] Taking the determination of the backward call sequence of the module under test by cloud service module 110 as an example: Cloud service module 110 obtains the backward call sequence through backward call analysis. The goal of backward call analysis is to extract call sequences ending with the target function (module under test). These call sequences represent various real-world use cases of the target function in the project and have a high chance of capturing the entire process of object construction used in the target method. For example, a function might create an object o and call the target function v with object o as a parameter. Backward call analysis can trace back to the object construction process. To achieve this goal, cloud service module 110 first constructs a call graph between the functions included in the project, and then extracts all call paths ending with the target method as backward call sequences. In short, cloud service module 110 determines the backward call sequence of the module under test (target function) by constructing a call graph and extracting the backward call sequence of the module under test based on the call graph.

[0070] Constructing a call graph refers to representing the call relationships between functions within the project containing the target function using a directed call graph. Specifically, the call graph includes multiple nodes connected by directed edges. The nodes in the call graph represent functions extracted from the project by traversing the Abstract Syntax Tree (AST), and the directed edges represent the call relationships between functions. Between two nodes, the function pointed to by the directed edge is the called function, and the function at the other end of the directed edge is the function that initiates the call.

[0071] For example, Figure 5 The diagram shows a simplified code snippet and its call graph, as follows: Figure 5As shown, the call graph contains a node `method1` (represented as `v1`) from `RelevantClass1`, and nodes `method2` (represented as `v2`) and `method3` (represented as `v3`) from `RelevantClass2`. Clearly, `v1` is called by both `v2` and `v3`, while `v2` is called only by `v3`. Optionally, for each extracted function, one or more of the declarations (definitions) or constructors of the classes associated with the function (such as the subclass to which the function belongs, its superclass, and the classes of the function's internal parameters) can also be recorded. This information is crucial because the extracted functions may belong to or interact with variables / methods from multiple different classes. Understanding where these elements come from is essential for LLM to construct valid objects.

[0072] Sequence extraction refers to extracting the backward call sequence for the target function based on the aforementioned directed call graph. Specifically, the cloud service module 110 first identifies all entry nodes in the directed call graph and uses a depth-first search (DFS) strategy to collect all paths that start from an entry node and end at the target function. If a loop is encountered during path extraction, traversal of that path is stopped to prevent the creation of infinite paths. It is worth noting that for each public function in the class, this embodiment can regard it as a complete path, and its entry method is the function itself.

[0073] For example, return Figure 5 In the example above, for the objective function v1, the cloud service module 110 will extract three paths: p1 = {v3→v2→v1}, p2 = {v3→v1}, and p3 = {v1}. This shows that there may be multiple backward call paths (e.g., p1 and p2) from the same entry function to the objective function. Optionally, to reduce computational burden, the cloud service module 110 can employ an intuitive filtering strategy to compress backward call sequences with the same entry function to the objective function. That is, only the shortest path is retained. For example, in this example, path p2 will be selected (i.e., p1 will be filtered out). Filtering out shorter and more direct call paths based on this strategy is easier for LLM to understand and learn, and can reduce computational burden and improve testing efficiency.

[0074] Example 2: The call analysis information includes the forward call sequence of the module under test.

[0075] Taking the determination of the forward call sequence of the module under test by the cloud service module 110 as an example: the cloud service module 110 performs forward call analysis, recursively collecting all functions related to the program code branches included in the target function. These extracted functions help interpret the semantics truly involved in the program code branch conditions. In other words, forward call analysis searches forward from the target function to other functions called by the target function, and the results of these functions determine the execution path of the target function. The goal of forward call analysis is to collect all possible return results of relevant external functions, thereby...

[0076] The following is combined Figure 6 The example in the document describes the forward analysis process in detail. Figure 6 The diagram shows a simplified code snippet and its forward call sequence, such as... Figure 6 As shown, assume the objective function v is Figure 6 In the example above, to extract a set of all relevant functions for the target method v, the cloud service module 110 first identifies the program code branch conditions and captures all variables (var1, param1) and functions (method2) contained in these conditional expressions. All involved variables and functions (such as method2) are added to the forward call sequence. For each involved variable, the cloud service module 110 recursively extracts the relevant assignment statements; recursion means tracing back to all relevant functions based on the variable. In this example, the value of var1 is determined by method1 (var1 = method1), and method1 accepts a parameter var0 determined by method0. Therefore, the assignments of these two functions are considered to be related to var1, and thus, the involved functions (method0 and method1) are added to the forward call sequence. That is, the forward call sequence includes the set of all functions related to the target function. A difference from the backward call sequence may be that in the forward call sequence, functions may not have a call relationship, so the arrangement of functions can be unordered. In this way, the cloud service module 110 retrieves all functions that may affect the results of program code branch conditions in the objective function, in order to help the LLM more fully understand the behavior of the conditional expression.

[0077] Example 3: The call analysis information includes the forward call sequence and the backward call sequence of the module under test.

[0078] Step 303: Obtain the test coverage information of the module under test.

[0079] Test coverage information is used to indicate the coverage of the program code in the module under test. For example, test coverage information indicates the covered branches of program code in the module under test, or it indicates the uncovered branches of program code in the module under test.

[0080] For example, test coverage information may include a set of existing test cases for the module under test. In one design, the set of existing test cases includes at least one existing test case. These existing test cases implicitly indicate the covered branches of program code in the module under test, guiding the LLM to generate different test cases.

[0081] Due to the high cost of using LLM, in order to reduce costs and increase efficiency, this application proposes that, in one implementation, the cloud service module 110 uses the test case generation method provided in this application and other traditional methods to generate test cases. For example, the test generation module 110 first uses a traditional method (such as Pynguin) to generate some test cases. When the traditional method cannot improve the test coverage within a set time range, the test case generation method provided in this application is then used to generate new test cases that are different from the existing test cases, so as to ensure cost-effectiveness.

[0082] Therefore, in the first example, the existing test cases include test cases generated using traditional methods. In one case, the initial test task (for program code branches that are easy to cover) is assigned to a traditional test case generation tool, resulting in test cases generated by traditional methods. These test cases are used to test the easily covered program code branches in the module under test, and the existing test case set includes these test cases. In the second example, the existing test cases include test cases generated using the test case generation method of this application and test cases generated using traditional methods. The second example, based on the first example, uses the test case generation method provided in the embodiments of this application to generate new test cases. These new test cases can cover program code branches that are difficult to cover using traditional methods, and the newly generated test cases are added to the existing test case set for use in subsequent iterations.

[0083] It should be understood that generating test cases using both traditional methods and the LLM-based test case generation method provided in this application does not imply a dependency between the two. Furthermore, it should be noted that the test case generation tool in the traditional method may not be deployed on the cloud service module 110. The cloud service module 110 can obtain existing test case sets from other devices, such as test cases generated by traditional methods or existing test case sets input by the tenant.

[0084] In another implementation, embodiments of this application support generating test cases using only the test case generation method provided in this application. Therefore, in the third example, existing test cases only include those generated using the test case generation method of this application. In one possible scenario, if the tenant does not input any existing test cases, the cloud service module 110 may construct a test case before initially generating test cases. This test case can be compiled correctly but will not actually generate coverage. It is used to guide the LLM on how to correctly generate new test cases, thereby ensuring the consistency of LLM input data (prompt information) in the absence of existing test cases. That is, at one stage, the set of existing test cases does not include test cases generated by traditional methods, nor does it include test cases generated by the test case generation method provided in this application; it only includes test cases constructed by the cloud service module 110.

[0085] Through the above design, these existing test cases can be used as examples to guide LLM to generate test cases for program code branches that are not yet covered, thereby improving overall testing efficiency. Furthermore, LLM can learn from the examples to generate test cases, thereby improving the legality and accuracy of the test cases generated by LLM.

[0086] It should be noted that step 303 is an optional step and is not mandatory. For example, if the prompt message does not include test coverage information for the module under test, step 303 can be omitted.

[0087] Step 304: Construct prompt information based on the test data of the module under test.

[0088] In one example, the test data includes the program code and call analysis information of the module under test; correspondingly, the first prompt information includes the program code and call analysis information of the module under test. In another example, the test data includes the program code, call analysis information, and test coverage information of the module under test; correspondingly, the first prompt information includes the program code, call analysis information, and test coverage information of the module under test.

[0089] To address situations where the called analysis information includes existing test sets, and to reduce the computational burden on LLM while avoiding input length limitations caused by including all existing test cases in the prompts, this application employs a sampling method. This method selects a diverse set of test cases from all existing test cases to form a minimal (or smaller) test case set, achieving a balance between effectiveness and efficiency. Specifically,

[0090] To achieve this goal, embodiments of this application provide a sampling method based on test coverage. Taking a backward call sequence as an example, for each backward call sequence of the target method obtained from backward call analysis, all existing test cases that call the first function (denoted as v0) in the backward call sequence are collected as candidate test cases (denoted as T). Next, the test case with the highest coverage is selected from the candidate test cases, and then the next test case that can achieve the highest incremental coverage is selected sequentially. This is an iterative process until no more test cases can increase the coverage. In this way, we can obtain a minimum set of test cases to cover all covered branches.

[0091] For example, suppose candidate test cases include T1, T2, T3, and T4. Given the coverage of each candidate test case, first select the test case with the highest coverage, let's say T1. Then, iterate through T2, T3, and T4, selecting the test case that covers a different branch of the program code than T1 and adds the highest coverage. For instance, if T1 has 30% coverage, T2 has 20%, T3 has 15%, and T4 has 10%, then we can add T2 to T1 for testing, resulting in a new assumed coverage of 40%. Adding T3 to T1 for testing yields 35% coverage. Adding T4 to T1 for testing yields 36% coverage. Based on this, select candidate test case T2 and add it to the existing test case set. Then, iterate through T3 and T4 again, selecting the test case that covers a different branch of the program code than T1 and T2 and adds the highest coverage. For example, adding T3 to T1 and T2 for testing yields a new assumed coverage of 40%. Testing with test case T4 overlaid with T1 and T2 yields a 41% coverage rate. Based on this, candidate test case T4 is added to the existing test case set, while candidate test case T3 is removed. This process is repeated until a minimum set of test cases is obtained. Finally, this minimum set of test cases is included in the error message.

[0092] Coverage can be measured by dividing the number of lines of code executed within a test case by the total number of lines of code included in the test case. The code included in the test case can be the code of the module under test. Alternatively, coverage can be measured by dividing the number of branches of code executed by the test case by the total number of branches of all code within the module under test. The specific method is not limited.

[0093] In summary, in a specific example, the prompt information includes the program code of the module under test and the call sequence (including forward call sequence and / or backward call sequence).

[0094] Step 305: Input the prompt information into LLM to obtain one or more test cases generated by LLM.

[0095] The cloud service module 110 inputs the prompt information into the LLM and obtains the test cases for the module under test (such as the objective function v) generated by the LLM.

[0096] Regarding the call sequence, taking backward call sequences as an example, in one strategy, the cloud service module 110 inputs the entire backward call sequence of the module under test into the LLM. Alternatively, in other strategies, the cloud service module 110 inputs a portion of the backward call sequence of the module under test (such as a single backward call sequence) into the LLM each time. For example, to select the backward call sequence of the target function v in each iteration, the cloud service module 110 first uses the sequence p = {v} (i.e., the entry method is v itself), because this is the most direct sequence for the LLM to call and understand the semantics of v. If this sequence does not exist (e.g., the target method is not a public method), or if the test cases generated using it do not cover new branches, an unused backward call sequence is randomly selected. The newly selected backward call sequence is then input into the LLM to summarize the functionality of the target function v, enabling the LLM to gain a deeper understanding of the function semantics. When there are no remaining backward call sequences, the test generation process for the target function v terminates.

[0097] In another specific example, the prompt information includes the program code of the module under test, the call sequence, and the minimum test set of existing test cases. Alternatively, the LLM subsequently generates one or more new test cases based on this prompt information. Each new test case is executed, yielding new test case coverage information. The cloud service module 110 adds these newly generated test cases, along with their coverage information, to the existing test case set and removes the used minimum test set (i.e., the minimum test set included in the prompt information) from the existing test case set, resulting in an updated existing test case set. Then, using the same method, a new minimum test set is determined from the updated existing test case set to construct the prompt information for the next round, ensuring diversity in future test case selection. This feedback helps generate more test cases with different behaviors.

[0098] It is worth noting that the various examples of prompts described above are merely illustrative and should not constitute a limitation of the embodiments of this application. The prompts in the embodiments of this application may also include other information, such as the context of the module under test. The sources for constructing the context of the module under test include two: 1) the class where the function under test resides; 2) the class where existing test cases reside. For example, the context information includes the class where the function under test resides, and / or, detailed information about the class where existing test cases reside, such as some attributes and functions defined in the class. This context information ensures that LLM generates more accurate test cases.

[0099] For example, the prompts can also include descriptions of existing test cases (such as those included in the minimum test suite), such as: "These examples (existing test cases or fake test cases) enter the target function through the selected function call sequence." Such descriptions help the LLM understand the intent of the generation, which is to cover different branches of program code in the target function according to the function call sequence. Subsequently, the prompts instruct the LLM to generate different test cases.

[0100] The LLM in this application embodiment can be a neural network model, such as chatGTP, including any existing and future language model, as long as the LLM can generate test cases based on the prompt information provided in this application embodiment. This application does not limit this.

[0101] Step 306: Use the test cases generated by LLM to test the module under test and obtain the test results for each test case.

[0102] Taking a test case as an example, the test result of the test case is used to indicate whether the functionality of the program code branch in the test module covered by the test case is correct. The program code branch covered by the test case refers to the program code that is executed when the test case is executed. Optionally, the test result of the test case may also include coverage information, indicating the coverage of the test case.

[0103] The cloud service module 110 can repeat the above operations, using LLM to perform multiple rounds of calculations, thereby obtaining multiple rounds of generated test cases.

[0104] In an alternative implementation, when the coverage of the test cases generated by the LLM meets the coverage requirements, the cloud service module 110 can return these test cases to the user, such as by sending them to the user through the cloud management platform 120, or by the user downloading these test case sets from the interface provided by the cloud management platform 120. Alternatively, the user can operate the terminal device to retrieve these test cases from the back-end storage 130, and use these test cases to test the test module. No specific limitations are imposed.

[0105] Through the above design, backward call analysis is used to extract the calling patterns of the function under test within the project, thereby revealing its usage and addressing the challenge of constructing complex objects. Forward call analysis extracts information about the external functions that the function under test depends on, helping the LLM to better understand the function's functionality, especially its complex branching logic.

[0106] Similar to cloud service scenarios, this application embodiment can also provide services via a web-based dialogue model, with implementation methods similar to those in cloud service scenarios. Besides cloud service scenarios and similar methods, the test case generation method provided in this application embodiment can also be deployed in other scenarios. For example, the test case generation method can be used as part of a code programming assistance tool (such as an IDE plugin). Alternatively, it can be deployed as a standalone test case generation tool. Or, it can be used as a programming assistance assistant for existing or potentially future large models, within a code generation tool based on the large model. Alternatively, it can be used as a separate large model product API, generating test cases through web page or app interaction.

[0107] Figure 7 This example illustrates a flowchart of a test case generation method implemented using an IDE plugin in programming software. The difference between this embodiment and a cloud service scenario is that the programming software stores the program code files of the software engineering project; therefore, the IDE plugin can obtain the program code of the module under test through the background without user input.

[0108] like Figure 7As shown, when a user selects the test case generation function in the IDE plugin, the plugin first collects the program code of the user-specified test module and the complete code of the project. Based on this, static code analysis is performed locally on the user's machine, including: dependency analysis, extraction of forward and backward call sequences; secondly, analysis of existing test cases to determine if any already written test cases exist. If so, a test case filtering and selection process is initiated to obtain a minimal test case set, ensuring that the minimal test case set and the complete test case set have the same test coverage. After static code analysis, all the information needed to build the prompts is obtained, therefore the prompt information (denoted as the first prompt information) is constructed, and the pre-trained language model is called through the backend API interface. The pre-trained language model generates test cases based on the given prompts. The generated test case (denoted as the first test case) can be directly returned to the user. At the same time, the first test case is also added to the minimum test case set. That is, the test coverage information of the module under test is updated based on the first test case, so that the user can use the function generated by the test case again. For example, new prompts (denoted as the second prompts) can be built based on the updated test coverage information. The LLM generates new test cases (such as the third test case) based on the second prompts. This ensures that the test case set is continuously expanded until the test coverage requirements are met.

[0109] Based on the same inventive concept as the method embodiments, this application also provides a computing device for performing the above-described... Figure 3 The method implementation example of the method behavior. Figure 8 As shown, in one example, the computing device 800 includes an acquisition module 801, a construction module 802, and a processing module 803. Optionally, it also includes an update module 804. Specifically, in the computing device 800, the modules are connected to each other through a communication path.

[0110] Module 801 is used to acquire test data of the modules under test included in the project to be tested. The test data includes the program code and call analysis information of the modules under test. The call analysis information indicates the call relationships between the modules under test and other functional modules included in the project. See details in [link to documentation]. Figure 3 The description of step 301 in the method embodiment will not be repeated here.

[0111] Module 802 is used to construct the first prompt information based on the test data. This first prompt information prompts the pre-trained language model to construct test cases based on call analysis information, where different functional modules call different branches of the program code in the module under test. See details in [link to documentation]. Figure 3 The description of step 304 in the method embodiment will not be repeated here.

[0112] Processing module 803 is used to input the first prompt information constructed from the test data into the pre-trained language model to obtain the first test cases of the module under test generated by the pre-trained model. The first test cases are used to send to the user or to test the module under test, and the obtained test results indicate whether the functionality of the program code branches covered by the first test cases is correct. See details in [link to documentation]. Figure 3 The description of step 305 in the method embodiment will not be repeated here.

[0113] In one possible implementation, the call analysis information includes the forward call sequence and / or backward call sequence of the module under test; wherein the forward call sequence indicates one or more other functional modules called by the module under test, and the backward call sequence indicates the call path of one or more other functional modules calling the module under test. See details... Figure 3 The description of step 302 in the method embodiment will not be repeated here.

[0114] In one possible implementation, the test data also includes test coverage information for the module under test. This test coverage information indicates which program code branches in the module under test are covered by at least one second test case. The second test case is a test case generated prior to the first test case. The first test case is used to test program code branches in the module under test that are not covered by the second test case. See details... Figure 3 The description of step 303 in the method embodiment will not be repeated here.

[0115] In one possible implementation, the test coverage information includes at least one second test case.

[0116] In one possible implementation, the update module 804 is used to update the test coverage information based on the first test case to obtain the updated test coverage information. The updated test coverage information is used to indicate the program code branches in the module under test that are covered by the first test case and at least one second test case.

[0117] The processing module 803 is also used to input the second prompt information into the pre-trained language model to obtain the third test case generated by the pre-trained language model; wherein, the second prompt information includes the updated test coverage information, and the third test case is used to test the uncovered program code branches in the module under test.

[0118] In one possible implementation, when the acquisition module 801 acquires the test data of the modules to be tested included in the project to be tested, it is specifically used to: provide the user with an application programming interface (API), the API being used to indicate multiple fields representing different data items included in the test data; and receive the test data sent by the user, the test data including multiple fields and data content entered by the user for each field.

[0119] In one possible implementation, when the acquisition module 801 acquires the test data of the modules to be tested included in the project to be tested, it is specifically used to: provide a configuration interface to the user; receive the test data input or selected by the user in the configuration interface, wherein the test data includes multiple data configuration items and the data content input or selected by the user for each data configuration item.

[0120] For example, the implementation of processing module 803 in computing device 800 will be described below. Similarly, the implementation of acquisition module 801 and update module 804 can refer to the implementation of processing module 803.

[0121] When implemented in software, the processing module 803 can be an application or code block running on a computer device. The computer device can be at least one of a physical host, virtual machine, container, or other computing device. Furthermore, there can be one or more computer devices. For example, the processing module 803 can be an application running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application can be distributed within the same availability zone (AZ) or in different AZs. Similarly, the multiple hosts / virtual machines / containers used to run the application can be distributed within the same region or in different regions. Typically, a region can include multiple AZs.

[0122] Similarly, multiple hosts / virtual machines / containers used to run the application can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a region can include multiple VPCs, and a VPC can include multiple Availability Zones (AZs).

[0123] When implemented in hardware, the processing module 803 may include at least one computing device, such as a server. Alternatively, the processing module 803 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0124] The processing module 803 includes multiple computing devices that can be distributed within the same Availability Zone (AZ) or in different AZs. Similarly, the processing module 803 can be distributed within the same region or in different regions. Likewise, the processing module 803 can be distributed within the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0125] It should be noted that the module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. The functional modules in this embodiment can be integrated into one module, or each module can exist physically separately, or two or more modules can be integrated into one module. For example, processing module 803 and update module 804 can be integrated into one module, or acquisition module 801 and construction module 802 can be the same module. The integrated units described above can be implemented in hardware or as software functional units.

[0126] This application also provides a computing device 900. For example... Figure 9 As shown, the computing device 900 includes a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate with each other via the bus 902. The computing device 900 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 900.

[0127] The 902 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 9 The bus 902 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 902 may include a path for transmitting information between various components of the computing device 900 (e.g., memory 906, processor 904, communication interface 908).

[0128] Processor 904 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0129] The memory 906 may include volatile memory, such as random access memory (RAM). The processor 904 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0130] The memory 906 stores executable program code, and the processor 904 executes the executable program code to implement the functions of the aforementioned acquisition module 801, construction module 802, processing module 803, and update module 804, thereby realizing the test case generation method. That is, the memory 906 stores instructions for the computing device 800 to execute the test case generation method provided in this application.

[0131] The communication interface 908 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 900 and other devices or communication networks.

[0132] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server. In some embodiments, the computing device may also be a desktop computer, a laptop computer, or a smartphone, or other terminal device.

[0133] like Figure 10 As shown, the computing device cluster includes at least one computing device 900. The memory 906 in one or more computing devices 900 in the computing device cluster may store the same instructions for executing test case generation methods.

[0134] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the test case generation method. In other words, a combination of one or more computing devices 900 can jointly execute the instructions for executing the test case generation method.

[0135] It should be noted that the memory 906 in different computing devices 900 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the data access device. That is, the instructions stored in the memory 906 of different computing devices 900 can implement the functions of one or more modules among the acquisition module 801, construction module 802, processing module 803, and update module 804.

[0136] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 11 One possible implementation is shown. For example... Figure 11 As shown, two computing devices 900A and 900B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 906 in computing device 900A stores instructions for executing the functions of the acquisition module 801 and the construction module 802. Simultaneously, the memory 906 in computing device 900B stores instructions for executing the functions of the processing module 803 and the update module 804.

[0137] It should be understood that Figure 11 The functions of the computing device 900A shown can also be performed by multiple computing devices 900. Similarly, the functions of the computing device 900B can also be performed by multiple computing devices 900.

[0138] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 10 and Figure 11The connection method of the computing device cluster is different in that the memory 906 of one or more computing devices 900 in the computing device cluster can store the same instructions for executing the test case generation method.

[0139] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the test case generation method. In other words, a combination of one or more computing devices 900 can jointly execute the instructions for executing the test case generation method.

[0140] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute a test case generation method, or instructs the computing device to execute a test case generation method.

[0141] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute a test case generation method, or instruct the computing device to execute a test case generation method.

[0142] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0143] The software implementation portion of the above embodiments can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0144] The various illustrative logic units and circuits described in the embodiments of this application can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0145] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, software units executed by a processor, or a combination of both. The software units can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative descriptions of the application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Thus, if such modifications and modifications fall within the scope of the claims and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A test case generation method, characterized in that, The method includes: Obtain test data of the modules to be tested included in the project to be tested. The test data includes the program code and call analysis information of the modules to be tested. The call analysis information is used to indicate the call relationship between the modules to be tested and other functional modules included in the project. Based on the test data, a first prompt message is constructed. The first prompt message is used to prompt the pre-trained language model to construct test cases based on the call analysis information, in which different functional modules call different program code branches in the module under test. The first prompt information is input into the pre-trained language model to obtain the first test case of the module under test generated by the pre-trained model; the first test case is used to send to the user or to test the module under test, and the obtained test result indicates whether the function of the program code branch covered by the first test case is correct.

2. The method as described in claim 1, characterized in that, The call analysis information includes the forward call sequence and / or backward call sequence of the module under test; wherein, the forward call sequence indicates one or more other functional modules called by the module under test, and the backward call sequence indicates the call path of one or more other functional modules calling the module under test.

3. The method as described in claim 1 or 2, characterized in that, The test data also includes test coverage information of the module under test. The test coverage information is used to indicate the program code branches in the module under test that are covered by at least one second test case. The second test case is a test case generated before the first test case. The first test case is used to test the program code branches in the module under test that are not covered by the second test case.

4. The method as described in claim 3, characterized in that, The test coverage information includes at least one second test case.

5. The method as described in claim 3 or 4, characterized in that, The method further includes: The test coverage information is updated based on the first test case to obtain updated test coverage information. The updated test coverage information is used to indicate the program code branches in the module under test that are covered by the first test case and the at least one second test case. The second prompt information is input into the pre-trained language model to obtain the third test case generated by the pre-trained language model; wherein, the second prompt information includes the updated test coverage information, and the third test case is used to test the uncovered program code branches in the module under test.

6. The method according to any one of claims 1-5, characterized in that, The acquisition of test data for the modules to be tested included in the project to be tested includes: Provide users with an application programming interface (API) for indicating multiple fields representing different data items included in the test data; The system receives the test data sent by the user, the test data including the multiple fields and the data content entered by the user for each field.

7. The method according to any one of claims 1-5, characterized in that, The acquisition of test data for the modules to be tested included in the project to be tested includes: Provide a configuration interface to users; The system receives the test data input or selected by the user on the configuration interface, wherein the test data includes multiple data configuration items and the data content input or selected by the user for each data configuration item.

8. The method according to any one of claims 1-7, characterized in that, The method is applied to a cloud management platform; the acquisition of test data for the modules to be tested included in the project to be tested includes: The cloud management platform determines the test data that the user inputs or selects on the cloud management platform; The method further includes: The cloud management platform sends a set of test cases to the user, and the set of test cases includes the second test case.

9. A test case generation device, characterized in that, The device includes: The acquisition module is used to acquire test data of the modules under test included in the project to be tested. The test data includes the program code and call analysis information of the modules under test. The call analysis information is used to indicate the call relationship between the modules under test and other functional modules included in the project. A construction module is used to construct a first prompt message based on the test data. The first prompt message is used to prompt the pre-trained language model to construct test cases based on the call analysis information, in which different functional modules call different program code branches in the module under test. The processing module is used to input the first prompt information constructed from the test data into the pre-trained language model to obtain the first test case of the module under test generated by the pre-trained model; the first test case is used to send to the user or to test the module under test, and the obtained test result indicates whether the function of the program code branch covered by the first test case is correct.

10. The apparatus as claimed in claim 9, characterized in that, The call analysis information includes the forward call sequence and / or backward call sequence of the module under test; wherein, the forward call sequence indicates one or more other functional modules called by the module under test, and the backward call sequence indicates the call path of one or more other functional modules calling the module under test.

11. The apparatus as claimed in claim 9 or 10, characterized in that, The test data also includes test coverage information of the module under test. The test coverage information is used to indicate the program code branches in the module under test that are covered by at least one second test case. The second test case is a test case generated before the first test case. The first test case is used to test the program code branches in the module under test that are not covered by the second test case.

12. The apparatus as claimed in claim 11, characterized in that, The test coverage information includes at least one second test case.

13. The apparatus as claimed in claim 11 or 12, characterized in that, The device also includes an update module: The update module is used to update the test coverage information based on the first test case to obtain updated test coverage information. The updated test coverage information is used to indicate the program code branches in the module under test that are covered by the first test case and the at least one second test case. The processing module is further configured to input the second prompt information into the pre-trained language model to obtain a third test case generated by the pre-trained language model; wherein the second prompt information includes the updated test coverage information, and the third test case is used to test the uncovered program code branches in the module under test.

14. The apparatus according to any one of claims 9-13, characterized in that, When acquiring test data of the modules to be tested included in the project to be tested, the acquisition module is specifically used to: provide the user with an application programming interface (API), the API being used to indicate multiple fields representing different data items included in the test data; and receive the test data sent by the user, the test data including the multiple fields and the data content input by the user for each field.

15. The apparatus according to any one of claims 9-14, characterized in that, When acquiring test data of the modules to be tested included in the project to be tested, the acquisition module is specifically used to: provide a configuration interface to the user; receive the test data input or selected by the user in the configuration interface, wherein the test data includes multiple data configuration items and the data content input or selected by the user for each data configuration item.

16. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 8.

17. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, Includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 8.

Images