Method and device for generating cascaded unit test cases with loosely coupled language models

Abstract

The present invention discloses a cascaded unit test case generation method and device with loosely coupled language models. Based on the code generation and instruction following capabilities of a large language model, unit tests are generated under regression settings, and the unit tests are ensured to be compilable and executable under the environment of the original warehouse, ultimately achieving a high coverage rate of the project under test. For the first time, the unit test generation task is split into the cascade generation tasks of test prefixes and test predictions, and context learning and retrieval enhancement techniques are used to perform generation that is not dependent on a specific large language model.

Description

Technical Field

[0001] The invention belongs to the field of intelligent software engineering, and in particular relates to a method and device for generating cascaded unit test cases with loose coupling of a language model. Background Art

[0002] End-to-end unit testing solutions can be divided into two categories: traditional methods and methods based on large language models (LLMs). Traditional methods include strategies based on search, randomization, or constraints, whose goal is to maximize test coverage (for example, Evosuite, Randoop, and Agitar). However, these methods have limited interpretability, especially method names and variables, which are often difficult to understand. In addition, these methods require a complete code repository and configuration environment for test case generation. The advantage of traditional methods is high coverage.

[0003] Recently, large language models have achieved great success in the field of natural language processing and can also be applied to software testing scenarios. Such methods can be further divided into two subcategories: methods based on pre-training and fine-tuning paradigms (e.g., AthenaTest, A3Test, and CAL-LM) and methods based on prompt engineering (e.g., CodaMosa, QTypist, ChatUniTest, and TestPilot). Although these methods have made great progress in comprehensibility and the test inputs are closer to the real data in actual applications, they are weaker than traditional methods in terms of test correctness and coverage.

[0004] After observing the handwritten unit tests of Java projects in Defects4J, a well-known benchmark dataset in the testing field, we found that a complete test case can often be divided into two parts: 1) test prefix: data preparation based on the actual function of the method under test, and constructing objects of the class to which the method under test belongs; 2) test oracle: complete the call of the method under test and verify whether the result after the call meets expectations. Therefore, the test case generation task is divided into a cascade generation task of first generating the test prefix and then generating the test oracle. Currently, a large number of language models have been trained on code knowledge (for example, gpt-3.5-turbo, codellama, deepseek-coder, codegeex), and have good code understanding and generation capabilities. However, since the task of cascading test case generation is still too new for such models that have not been specially optimized, specially constructed prompt words are needed to guide it. In addition, providing high-quality (diverse and similar) samples is also crucial for the model.

[0005] In addition, after the test method is embedded in the test class after cascade generation, it is still necessary to enter the feedback stage. The feedback stage mainly needs to verify whether the test class obtained in the generation stage can be compiled and successfully executed in the real environment. If the result is negative, it is necessary to collect relevant error reporting knowledge, and then provide the context and original case again for repair. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies of the prior art and to provide a method and device for generating cascaded unit test cases with loose coupling of language models.

[0007] The objective of the present invention is achieved through the following technical solution: a method for generating cascaded unit test cases with loosely coupled language models, comprising the following steps:

[0008] (1) Extract all existing unit test cases from the original repository of the project under test using a script, and then parse the method under test of each existing unit test case through the javaparser parser. After splitting the existing unit test cases, a test prefix sample pool consisting of the method under test-data preparation pairs and a test prediction sample pool consisting of the method under test-call verification pairs are formed;

[0009] (2) For any target method that needs to generate a test case, construct the corresponding prompt word and input it into the large language model to generate the test prefix and test prediction in turn;

[0010] (3) After assembling the test prefix and the test prediction into a complete test method, the complete test method is then embedded into the test class template to obtain a complete test class;

[0011] (4) Compile and execute the complete test class. If there is a failure, collect relevant information to enable the large language model to perform corresponding compilation or execution repair. When there are no errors in the compilation and execution results, the test class can be used as the final unit test and the final unit test is added to the seed test case set;

[0012] (5) Use the coverage testing tool jacoco to perform the final unit test and obtain the bytecode instruction coverage and branch coverage of the final unit test on the corresponding target method; at the same time, set the generation counter and coverage stagnation counter;

[0013] (6) The seed test case set and the corresponding target method are taken as input, and the bytecode instruction coverage, branch coverage, generation counter count and coverage stagnation counter count are combined to select the mutation fuzz testing process or the generative fuzz testing process to obtain the final generation result of the target method and return it to the user.

[0014] Furthermore, the step (1) specifically includes the following sub-steps:

[0015] (1.1) Use regular expressions to match all test cases containing @Test annotations from the original repository of the project under test as existing unit test cases, and then extract the matching test cases;

[0016] (1.2) Use the javaparser parser to parse the business code of the original repository of the project under test to obtain the mapping set of method signatures and complete methods;

[0017] (1.3) Use the javaparser to parse the method signature of each test case to obtain the method signature mapping set between the test case and the method being tested;

[0018] (1.4) Associating the mapping set of method signatures and complete methods through test cases and method signature mapping sets of the tested methods to form multiple pairs of tested methods and unit test cases;

[0019] (1.5) Split each existing unit test case into data preparation and call verification of the tested method, and construct a test prefix sample pool consisting of multiple tested method-data preparation pairs and a test oracle sample pool consisting of multiple tested method-call verification pairs with the tested method respectively.

[0020] Furthermore, the step (2) specifically includes the following sub-steps:

[0021] (2.1) For any target method for which a test case needs to be generated, extract the context of the target method, where the context includes the class name and constructor of the test class corresponding to the target method;

[0022] (2.2) Use UniXCoder to encode each method-data preparation pair in the test prefix sample pool into a corresponding prefix pair vector to obtain a set of prefix pair vectors;

[0023] (2.3) Use UniXCoder to encode each method-call verification pair in the test oracle sample pool into a corresponding oracle pair vector to obtain a set of oracle pair vectors;

[0024] (2.4) Perform KMeans clustering on the set of prefix pair vectors to obtain N number of prefix pair vector clusters; and perform KMeans clustering on the set of prediction pair vectors to obtain N number of prediction pair vector clusters; where N=5;

[0025] (2.5) Use UniXCoder to encode the code of the target method to obtain the target method vector;

[0026] (2.6) Calculate the cosine similarity of the target method vector and all the prefix pair vectors in each prefix pair vector cluster, select M prefix pair vectors with the largest cosine similarity from each prefix pair vector cluster, and screen out the corresponding method under test-data preparation pairs based on the selected prediction pair vectors; Calculate the cosine similarity of the target method vector and all the prediction pair vectors in each prediction pair vector cluster, select M prediction pair vectors with the largest cosine similarity from each prediction pair vector cluster, and screen out the corresponding method under test-call verification pairs based on the selected prediction pair vectors; where M=2;

[0027] (2.7) The extracted context of the target method and all the filtered tested method-data preparation pairs are embedded into the defined prompt word template, and the large language model is called for reasoning to obtain the test prefix; and the extracted context of the target method and all the filtered tested method-call verification pairs are embedded into the defined prompt word template, and the large language model is called for reasoning to obtain the test prediction.

[0028] Furthermore, the step (3) specifically includes the following sub-steps:

[0029] (3.1) injecting the test prefix into the test method template to obtain a temporary test method, and retaining a placeholder for the test prediction after the temporary test method;

[0030] (3.2) Use the test oracle to replace the placeholders in the temporary test method and assemble it into a complete test method;

[0031] (3.3) Delete the test methods from all the test classes in the original repository that are in the same package as the target method, merge and remove the remaining contents, and retain the test method placeholder to obtain the test class template;

[0032] (3.4) Use the complete test method to replace the placeholder in the test class template to obtain a complete test class.

[0033] Furthermore, the step (4) specifically includes the following sub-steps:

[0034] (4.1) Move the complete test class to the corresponding directory of the original warehouse;

[0035] (4.2) Use the javac command to compile the moved complete test class. If it contains compilation errors, input the compilation error information and the complete test class into the large language model, so that the large language model generates a repaired test class. If it does not contain compilation errors, the complete test class is the test class that has passed compilation, and execute step (4.4);

[0036] (4.3) Use the javac command to compile the test class after the large language model is compiled and repaired. If the test class after the large language model is repaired still contains compilation errors, repeat step (4.2) until the compilation passes to obtain a compiled test class, or the first repair upper limit is reached and the generation of the target method is abandoned; wherein the first repair upper limit is 4;

[0037] (4.4) Use the java command to execute the compiled test class. If it contains execution errors, input the execution error information and the compiled test class into the large language model, so that the large language model generates the test class after the execution repair. If it does not contain execution errors, the compiled test class is used as the executed test class and the final unit test, and step (4.5) is no longer executed.

[0038] (4.5) If the test class after the repair still contains execution errors, repeat step (4.4) until it no longer contains errors after execution. The repaired test class is then used as the test class that passes the execution and as the final unit test, or the second repair limit is reached and the generation of the target method is abandoned; the second repair limit is 3.

[0039] Furthermore, the step (5) specifically includes the following sub-steps:

[0040] (5.1) Configure the Jacoco-Maven plugin in the project object model file of the original repository and set the execution identifier to test;

[0041] (5.2) Use the coverage testing tool jacoco to execute the final unit test. When the execution is successful, the coverage result file is obtained in the target / site path in the root directory of the original warehouse;

[0042] (5.3) For the coverage result file, first locate the class label of the target method, then locate the label of the target method, and obtain the bytecode instruction coverage and branch coverage of the final unit test on the corresponding target method;

[0043] (5.4) Set the initial value of the generation counter to 1 and the maximum value to 20, and set the initial value of the coverage stagnation counter to 0 and the maximum value to 5.

[0044] Furthermore, the step (6) specifically includes the following sub-steps:

[0045] (6.1) First, determine the current bytecode instruction coverage and branch coverage: when the current bytecode instruction coverage is not less than 90% and the current branch coverage is not less than 100%, directly use the current unit test case as the final generation result of the corresponding target method and return it to the user; otherwise, determine the count size of the current generation counter: if the count of the current generation counter is equal to 20, directly use the current unit test case as the final generation result of the corresponding target method and return it to the user; if the count of the current generation counter is less than 20, execute step (6.2);

[0046] (6.2) Then determine the size of the current bytecode instruction coverage and branch coverage: if the current bytecode instruction coverage and branch coverage are equal to the previous bytecode instruction coverage and branch coverage, then add 1 to the current coverage stall counter and execute step (6.3); if the current bytecode instruction coverage is greater than the previous bytecode instruction coverage or the current branch coverage is greater than the previous branch coverage, then reset the coverage stall counter to 0 and choose to enter the variant fuzz testing process; otherwise, abandon the generation of the target method;

[0047] (6.3) Then determine the current count size of the coverage stagnation counter: if the current count size of the coverage stagnation counter is less than 5, then choose to enter the mutation fuzz testing process; otherwise, choose to enter the generative fuzz testing process.

[0048] Furthermore, the variant fuzz testing process specifically includes the following sub-steps:

[0049] (a.1) Embed the current seed test case set into Java markdown syntax, connect the source code of the target method, and add mutation instructions to obtain the mutation fuzzy test prompt words;

[0050] (a.2) then inputting the variant fuzzy test prompt word into the large language model to obtain a variant fuzzy test output;

[0051] (a.3) extracting a complete mutation test class from the mutation fuzz test output, and then executing step (4) using the complete mutation test class;

[0052] (a.4) Increase the count of the current generation counter by 1;

[0053] (a.5) After executing step (4), if a new final unit test is obtained, the new final unit test is added to the seed test case set and the process returns to step (5).

[0054] Furthermore, the generative fuzz testing process specifically includes the following sub-steps:

[0055] (b.1) Embed the current seed test case set into Java markdown syntax, connect the source code of the target method, and add generation instructions to obtain the generative fuzzy test prompt words;

[0056] (b.2) then inputting the generative fuzzy test prompt word into the large language model to obtain a generative fuzzy test output;

[0057] (b.3) extracting a complete fuzz test class from the generated fuzz test output, and then executing step (4) using the complete fuzz test class;

[0058] (b.4) Increase the count of the current generation counter by 1;

[0059] (b.5) After executing step (4), if a new final unit test is obtained, the new final unit test is added to the seed test case set and the process returns to step (5).

[0060] The present invention also includes a cascaded unit test case generation device with loosely coupled language models, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used for the above-mentioned cascaded unit test case generation method with loosely coupled language models.

[0061] The present invention also includes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned method for generating cascaded unit test cases with loose coupling of a language model is implemented.

[0062] The beneficial effects of the present invention are as follows: CasModaTest can timely generate test cases with high accuracy and input close to the actual call logic during the development process, discover potential bugs as early as possible, improve the quality of software engineering, and reduce manpower investment in the testing process; in addition, the present invention has realized the automation of AST extraction in the process of building a sample library using retrieval enhancement generation, and supports multiple types of large code models, so it can be highly flexibly reused in other projects for unit test generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of a method for generating cascaded unit test cases for a loosely coupled language model;

[0064] Figure 2 Structural diagram of a loosely coupled cascaded unit test case generation setup for a language model. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical scheme and advantages of the present invention more clear, the present invention is further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0066] Example 1: Figure 1 As shown, the present invention provides a method for generating cascaded unit test cases for a language model loosely coupled, comprising the following sub-steps:

[0067] (1) Use scripts to extract all existing unit test cases from the original repository of the project under test, and then use the javaparser to parse the tested method of each existing unit test case. After splitting the existing unit test cases, a test prefix sample pool consisting of the tested method-data preparation pairs (focal function-test prefix pairs) and a test oracle sample pool consisting of the tested method-call verification pairs (focal function-test oracle pairs) are formed.

[0068] The step (1) specifically includes the following sub-steps:

[0069] (1.1) Use regular expressions to match all test cases containing the @Test annotation from the original repository of the project under test as existing unit test cases, and then extract the matching test cases.

[0070] (1.2) Use the javaparser parser to parse the business code of the original repository of the project under test to obtain a mapping set of method signatures and complete methods.

[0071] (1.3) Use the javaparser parser to parse the method signature of the method under test for each test case, and obtain the method signature mapping set between the test case and the method under test.

[0072] (1.4) Associate the mapping set of method signatures and complete methods through test cases and method signature mapping sets of the method under test to form multiple pairs of method under test and unit test cases.

[0073] (1.5) Split each existing unit test case into data preparation and call verification of the tested method, and construct a test prefix sample pool consisting of multiple tested method-data preparation pairs and a test oracle sample pool consisting of multiple tested method-call verification pairs with the tested method respectively.

[0074] (2) For any target method that needs to generate a test case, construct the corresponding prompt word and input it into the large language model to generate a test prefix and a test oracle in turn.

[0075] The step (2) specifically includes the following sub-steps:

[0076] (2.1) For any target method for which a test case needs to be generated, extract the context of the target method, where the context includes the class name and constructor of the test class corresponding to the target method.

[0077] (2.2) Use UniXCoder to encode each method-data preparation pair in the test prefix sample pool into a corresponding prefix pair vector to obtain a set of prefix pair vectors.

[0078] (2.3) Use UniXCoder to encode each method-call verification pair in the test oracle sample pool into a corresponding oracle pair vector to obtain a set of oracle pair vectors.

[0079] (2.4) Perform KMeans clustering on the set of prefix pair vectors to obtain N number of prefix pair vector clusters; and perform KMeans clustering on the set of prediction pair vectors to obtain N number of prediction pair vector clusters; where N=5.

[0080] (2.5) Use UniXCoder to encode the code of the target method to obtain the target method vector.

[0081] (2.6) Calculate the cosine similarity of the target method vector with all the prefix pair vectors in each prefix pair vector cluster, select M prefix pair vectors with the largest cosine similarity from each prefix pair vector cluster, and screen out the corresponding method under test-data preparation pairs based on the selected prediction pair vectors; calculate the cosine similarity of the target method vector with all the prediction pair vectors in each prediction pair vector cluster, select M prediction pair vectors with the largest cosine similarity from each prediction pair vector cluster, and screen out the corresponding method under test-call verification pairs based on the selected prediction pair vectors; where M=2.

[0082] (2.7) The extracted context of the target method and all the filtered tested method-data preparation pairs are embedded into the defined prompt word template, and the large language model is called for reasoning to obtain the test prefix; and the extracted context of the target method and all the filtered tested method-call verification pairs are embedded into the defined prompt word template, and the large language model is called for reasoning to obtain the test prediction.

[0083] (3) After assembling the test prefix and test prediction into a complete test method, the complete test method is then embedded into the test class template to obtain a complete test class.

[0084] The step (3) specifically includes the following sub-steps:

[0085] (3.1) Inject the test prefix into the test method template to obtain a temporary test method, and reserve a placeholder for the test prediction after the temporary test method.

[0086] (3.2) Use the test oracle to replace the placeholders in the temporary test method and assemble it into a complete test method.

[0087] (3.3) Delete the test methods from all test classes in the original repository that are in the same package as the target method, merge and deduplicate the remaining contents, and retain the test method placeholder to obtain the test class template.

[0088] (3.4) Use the complete test method to replace the placeholder in the test class template to obtain a complete test class.

[0089] (4) Compile and execute the complete test class. If there is a failure, collect relevant information to enable the large language model to perform corresponding compilation or execution repairs. When there are no errors in the compilation and execution results, the test class can be used as the final unit test and the final unit test is added to the seed test case set.

[0090] The step (4) specifically includes the following sub-steps:

[0091] (4.1) Move the complete test class to the corresponding directory of the original repository.

[0092] (4.2) Use the javac command to compile the complete test class after the move. If it contains compilation errors, input the compilation error information and the complete test class into the large language model, so that the large language model generates a repaired test class; if it does not contain compilation errors, the complete test class is the test class that has passed the compilation, and execute step (4.4).

[0093] (4.3) Use the javac command to compile the test class after the large language model is compiled and repaired. If the test class after the large language model is repaired still contains compilation errors, repeat step (4.2) until the compilation passes to obtain a compiled test class, or reach the first repair upper limit and give up generating the target method; wherein the first repair upper limit is 4.

[0094] (4.4) Use the java command to execute the compiled test class. If it contains execution errors, input the execution error information and the compiled test class into the large language model, so that the large language model generates the test class after the execution repair. If it does not contain execution errors, the compiled test class is used as the executed test class and as the final unit test, and step (4.5) is no longer executed.

[0095] (4.5) If the test class after the repair still contains execution errors, repeat step (4.4) until it no longer contains errors after execution. The repaired test class is then used as the test class that passes the execution and as the final unit test, or the second repair limit is reached and the generation of the target method is abandoned; the second repair limit is 3.

[0096] (5) Use the coverage testing tool jacoco to perform the final unit test and obtain the bytecode instruction coverage and branch coverage of the final unit test on the corresponding target method; at the same time, set the generation counter and coverage stagnation counter.

[0097] The step (5) specifically includes the following sub-steps:

[0098] (5.1) Configure the Jacoco-Maven plugin in the project object model file (pom.xml) of the original repository and set the execution identifier to test, thus preparing the environment for using the coverage testing tool jacoco later.

[0099] (5.2) Use the coverage testing tool jacoco to execute the final unit test. When the execution is successful, the coverage result file is obtained in the target / site path in the root directory of the original warehouse.

[0100] (5.3) For the coverage result file, first locate the class label of the target method, then locate the label of the target method, and obtain the bytecode instruction coverage and branch coverage of the final unit test on the corresponding target method. Among them, the bytecode instruction coverage is used as the baseline bytecode instruction coverage, and the branch coverage is used as the baseline branch coverage.

[0101] (5.4) According to the difficulty of generating the final unit test of the original repository, set the initial value of the generation counter to 1 and the maximum value to 20, and set the initial value of the coverage stagnation counter to 0 and the maximum value to 5.

[0102] (6) The seed test case set and the corresponding target method are taken as input, and the bytecode instruction coverage, branch coverage, generation counter count and coverage stagnation counter count are combined to select the mutation fuzz testing process or the generative fuzz testing process to obtain the final generation result of the target method and return it to the user.

[0103] The step (6) specifically includes the following sub-steps:

[0104] (6.1) First, determine the current bytecode instruction coverage and branch coverage: when the current bytecode instruction coverage is not less than 90% and the current branch coverage is not less than 100%, directly use the current unit test case as the final generation result of the corresponding target method and return it to the user; otherwise, determine the count size of the current generation counter: if the count of the current generation counter is equal to 20, directly use the current unit test case as the final generation result of the corresponding target method and return it to the user; if the count of the current generation counter is less than 20, execute step (6.2).

[0105] (6.2) Then determine the size of the current bytecode instruction coverage and branch coverage: if the current bytecode instruction coverage and branch coverage are equal to the previous bytecode instruction coverage and branch coverage, then the current coverage stagnation counter count is increased by 1 and step (6.3) is executed; if the current bytecode instruction coverage is greater than the previous bytecode instruction coverage or the current branch coverage is greater than the previous branch coverage, then the coverage stagnation counter count is reset to 0 and the variant fuzz testing process is selected to enter; otherwise, the generation of the target method is abandoned.

[0106] (6.3) Then determine the current count size of the coverage stagnation counter: if the current count size of the coverage stagnation counter is less than 5, then choose to enter the mutation fuzz testing process; otherwise, choose to enter the generative fuzz testing process.

[0107] The variant fuzz testing process specifically includes the following sub-steps:

[0108] (a.1) Embed the current seed test case set into the Java markdown syntax, connect the source code of the target method, and add the mutation instruction to obtain the mutation fuzzy test prompt. The mutation instruction is Please generate a mutated test case to cover more points of the focal method.

[0109] (a.2) The variant fuzzy test prompt word is then input into the large language model to obtain a variant fuzzy test output.

[0110] (a.3) Extracting a complete mutation test class from the mutation fuzz test output, and then executing step (4) using the complete mutation test class.

[0111] (a.4) Increase the count of the current generation counter by 1.

[0112] (a.5) After executing step (4), if a new final unit test is obtained, the new final unit test is added to the seed test case set and the process returns to step (5).

[0113] The generative fuzz testing process specifically includes the following sub-steps:

[0114] (b.1) Embed the current seed test case set into the Java markdown syntax, connect the source code of the target method, and add the generation instruction to obtain the generative fuzzy test prompt. The generation instruction is Please generate a strong test case to cover as many points of the focal method as possible.

[0115] (b.2) The generative fuzzy test prompt word is then input into the large language model to obtain a generative fuzzy test output.

[0116] (b.3) Extracting a complete fuzz test class from the generative fuzz test output, and then executing step (4) using the complete fuzz test class.

[0117] (b.4) Increase the count of the current generation counter by 1.

[0118] (b.5) After executing step (4), if a new final unit test is obtained, the new final unit test is added to the seed test case set and the process returns to step (5).

[0119] Experimental results: The performance of five Java projects (Cli, Csv, Gson, Chart and Lang) of the unit test generation benchmark dataset Defects4J and five representative unit test generation tools (AthenaTest, A3Test, ChatUniTest, CAT-LM and CasModaTest (gpt3.5)) were compared. The performance indicators selected were the accuracy of the final generated results of the target method and the method coverage (focal method coverage) of the tested method. The results are shown in Tables 1 and 2.

[0120] Table 1: Comparison of the accuracy of the final generated results of the target method

[0121]

[0122] Table 2: Comparison of method coverage of the tested methods

[0123]

[0124] The impact of different types of large language models (CasModaTest (gpt4.0), CasModaTest (gpt3.5), CasModaTest (ds), CasModaTest (phind-cl), CasModaTest (cl), ChatUniTest, A3Test, AthenaTest and CAT-LM) on tool performance was explored on five Java projects (Cli, Csv, Gson, Chart and Lang) of the benchmark dataset Defects4J generated by unit tests. The performance indicators selected were the accuracy of the generated test cases and the focal method coverage of the tested methods. The results are shown in Tables 3 and 4.

[0125] Table 3: The impact of different large language models on the accuracy of test cases

[0126]

[0127] Table 4: The impact of different large language models on the method coverage of the tested methods

[0128]

[0129] Embodiment 2: Corresponding to the embodiment 1 of the aforementioned method for generating cascaded unit test cases with loose coupling of language models, the present invention further provides an embodiment of an apparatus for generating cascaded unit test cases with loose coupling of language models.

[0130] See also Figure 2 An embodiment of the present invention provides a cascaded unit test case generation device with loosely coupled language models, comprising one or more processors, for implementing a cascaded unit test case generation method with loosely coupled language models in the above embodiment.

[0131] An embodiment of a cascaded unit test case generation device with a loosely coupled language model of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From a hardware perspective, if Figure 2As shown, it is a hardware structure diagram of any device with data processing capability where a cascade unit test case generating device for a language model loosely coupled according to the present invention is located, except Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0132] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0133] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0134] The embodiment of the present invention also provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, a cascade unit test case generation method for a language model loosely coupled in the above embodiment is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capability described in any of the above embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Further, the computer-readable storage medium can also include both an internal storage unit and an external storage device of any device with data processing capability. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and can also be used to temporarily store data that has been output or is to be output.

[0135] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating cascaded unit test cases with loosely coupled language models, characterized in that: The following steps are involved: (1) Extract all existing unit test cases from the original repository of the project under test using a script, and then parse the method under test of each existing unit test case through the javaparser parser. After splitting the existing unit test cases, a test prefix sample pool consisting of the method under test-data preparation pairs and a test prediction sample pool consisting of the method under test-call verification pairs are formed; (2) For any target method that needs to generate a test case, construct the corresponding prompt word and input it into the large language model to generate the test prefix and test prediction in turn; (3) After assembling the test prefix and the test prediction into a complete test method, the complete test method is then embedded into the test class template to obtain a complete test class; (4) Compile and execute the complete test class. If there is a failure, collect relevant information to enable the large language model to perform corresponding compilation or execution repair. When there are no errors in the compilation and execution results, the test class can be used as the final unit test and the final unit test is added to the seed test case set; (5) Use the coverage testing tool jacoco to perform the final unit test and obtain the bytecode instruction coverage and branch coverage of the final unit test on the corresponding target method; at the same time, set the generation counter and coverage stagnation counter; (6) The seed test case set and the corresponding target method are taken as input, and the bytecode instruction coverage, branch coverage, generation counter count and coverage stagnation counter count are combined to select the mutation fuzz testing process or the generative fuzz testing process to obtain the final generation result of the target method and return it to the user.

2. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 1, characterized in that: The step (1) specifically includes the following sub-steps: (1.1) Use regular expressions to match all test cases containing @Test annotations from the original repository of the project under test as existing unit test cases, and then extract the matching test cases; (1.2) Use the javaparser parser to parse the business code of the original repository of the project under test to obtain the mapping set of method signatures and complete methods; (1.3) Use the javaparser to parse the method signature of each test case to obtain the method signature mapping set between the test case and the method being tested; (1.4) Associating the mapping set of method signatures and complete methods through test cases and method signature mapping sets of the tested methods to form multiple pairs of tested methods and unit test cases; (1.5) Split each existing unit test case into data preparation and call verification of the tested method, and construct a test prefix sample pool consisting of multiple tested method-data preparation pairs and a test oracle sample pool consisting of multiple tested method-call verification pairs with the tested method respectively.

3. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 2, characterized in that: The step (2) specifically includes the following sub-steps: (2.1) For any target method for which a test case needs to be generated, extract the context of the target method, where the context includes the class name and constructor of the test class corresponding to the target method; (2.2) Use UniXCoder to encode each method-data preparation pair in the test prefix sample pool into a corresponding prefix pair vector to obtain a set of prefix pair vectors; (2.3) Use UniXCoder to encode each method-call verification pair in the test oracle sample pool into a corresponding oracle pair vector to obtain a set of oracle pair vectors; (2.4) Perform KMeans clustering on the set of prefix pair vectors to obtain N number of prefix pair vector clusters; and perform KMeans clustering on the set of prediction pair vectors to obtain N number of prediction pair vector clusters; where N=5; (2.5) Use UniXCoder to encode the code of the target method to obtain the target method vector; (2.6) Calculate the cosine similarity of the target method vector with all the prefix pair vectors in each prefix pair vector cluster, select M prefix pair vectors with the largest cosine similarity from each prefix pair vector cluster, and screen out the corresponding method under test-data preparation pairs based on the selected prefix pair vectors; calculate the cosine similarity of the target method vector with all the prediction pair vectors in each prediction pair vector cluster, select M prediction pair vectors with the largest cosine similarity from each prediction pair vector cluster, and screen out the corresponding method under test-call verification pairs based on the selected prediction pair vectors; where M=2; (2.7) The extracted context of the target method and all the filtered tested method-data preparation pairs are embedded into the defined prompt word template, and the large language model is called for reasoning to obtain the test prefix; and the extracted context of the target method and all the filtered tested method-call verification pairs are embedded into the defined prompt word template, and the large language model is called for reasoning to obtain the test prediction.

4. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 3, characterized in that: The step (3) specifically includes the following sub-steps: (3.1) injecting the test prefix into the test method template to obtain a temporary test method, and retaining a placeholder for the test prediction after the temporary test method; (3.2) Use the test oracle to replace the placeholders in the temporary test method and assemble it into a complete test method; (3.3) Delete the test methods from all the test classes in the original repository that are in the same package as the target method, merge and remove the remaining contents, and retain the test method placeholder to obtain the test class template; (3.4) Use the complete test method to replace the placeholder in the test class template to obtain a complete test class.

5. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 4, characterized in that: The step (4) specifically includes the following sub-steps: (4.1) Move the complete test class to the corresponding directory of the original warehouse; (4.2) Use the javac command to compile the complete test class after the move. If there are compilation errors, input the compilation error information and the complete test class into the large language model, so that the large language model generates the repaired test class; If there are no compilation errors, the complete test class is a compiled test class and step (4.4) is executed; (4.3) Use the javac command to compile the test class after the large language model is compiled and repaired. If the test class after the large language model is repaired still contains compilation errors, repeat step (4.2) until the compilation passes to obtain a compiled test class, or the first repair upper limit is reached and the generation of the target method is abandoned; wherein the first repair upper limit is 4; (4.4) Use the java command to execute the compiled test class. If it contains execution errors, input the execution error information and the compiled test class into the large language model, so that the large language model generates the test class after the execution repair. If it does not contain execution errors, the compiled test class is used as the executed test class and the final unit test, and step (4.5) is no longer executed. (4.5) If the test class after the repair still contains execution errors, repeat step (4.4) until it no longer contains errors after execution. The repaired test class is then used as the test class that passes the execution and as the final unit test, or the second repair limit is reached and the generation of the target method is abandoned; the second repair limit is 3.

6. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 5, characterized in that: The step (5) specifically includes the following sub-steps: (5.1) Configure the Jacoco-Maven plugin in the project object model file of the original repository and set the execution identifier to test; (5.2) Use the coverage testing tool jacoco to execute the final unit test. When the execution is successful, the coverage result file is obtained in the target / site path in the root directory of the original warehouse; (5.3) For the coverage result file, first locate the class label of the target method, then locate the label of the target method, and obtain the bytecode instruction coverage and branch coverage of the final unit test on the corresponding target method; (5.4) Set the initial value of the generation counter to 1 and the maximum value to 20, and set the initial value of the coverage stagnation counter to 0 and the maximum value to 5.

7. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 6, characterized in that: The step (6) specifically includes the following sub-steps: (6.1) First, determine the current bytecode instruction coverage and branch coverage: when the current bytecode instruction coverage is not less than 90% and the current branch coverage is not less than 100%, directly use the current unit test case as the final generation result of the corresponding target method and return it to the user; otherwise, determine the count size of the current generation counter: if the count of the current generation counter is equal to 20, directly use the current unit test case as the final generation result of the corresponding target method and return it to the user; if the count of the current generation counter is less than 20, execute step (6.2); (6.2) Then determine the size of the current bytecode instruction coverage and branch coverage: if the current bytecode instruction coverage and branch coverage are equal to the previous bytecode instruction coverage and branch coverage, then add 1 to the current coverage stall counter and execute step (6.3); if the current bytecode instruction coverage is greater than the previous bytecode instruction coverage or the current branch coverage is greater than the previous branch coverage, then reset the coverage stall counter to 0 and choose to enter the variant fuzz testing process; otherwise, abandon the generation of the target method; (6.3) Then determine the current count size of the coverage stagnation counter: if the current count size of the coverage stagnation counter is less than 5, then choose to enter the mutation fuzz testing process; otherwise, choose to enter the generative fuzz testing process.

8. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 7, characterized in that: The variant fuzz testing process specifically includes the following sub-steps: (a.1) Embed the current seed test case set into Java markdown syntax, connect the source code of the target method, and add mutation instructions to obtain the mutation fuzzy test prompt words; (a.2) then inputting the variant fuzzy test prompt word into the large language model to obtain a variant fuzzy test output; (a.3) extracting a complete mutation test class from the mutation fuzz test output, and then executing step (4) using the complete mutation test class; (a.4) Increase the count of the current generation counter by 1; (a.5) After executing step (4), if a new final unit test is obtained, the new final unit test is added to the seed test case set and the process returns to step (5).

9. The method for generating cascaded unit test cases for a language model loosely coupled according to claim 8, characterized in that: The generative fuzz testing process specifically includes the following sub-steps: (b.1) Embed the current seed test case set into Java markdown syntax, connect the source code of the target method, and add generation instructions to obtain the generative fuzzy test prompt words; (b.2) then inputting the generative fuzzy test prompt word into the large language model to obtain a generative fuzzy test output; (b.3) extracting a complete fuzz test class from the generated fuzz test output, and then executing step (4) using the complete fuzz test class; (b.4) Increase the count of the current generation counter by 1; (b.5) After executing step (4), if a new final unit test is obtained, the new final unit test is added to the seed test case set and the process returns to step (5).

10. A cascaded unit test case generation device with loosely coupled language models, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a language model loosely coupled cascade unit test case generation method as described in any one of claims 1-9.

11. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the method for generating cascaded unit test cases for a language model loosely coupled as described in any one of claims 1-9 is implemented.

Images