Test case generation method and device, equipment and medium

Abstract

The invention discloses a test case generation method and device, equipment and a medium, and relates to the technical field of software test optimization. According to the scheme, the multiple initial test cases are obtained, and the test reports corresponding to the initial test cases are generated through the first preset test tool, so that the time for manually compiling the test cases is effectively shortened; when it is determined that a to-be-supplemented test case of which the corresponding code coverage rate index does not meet a preset requirement exists according to each test report, code uncovered information of the to-be-supplemented test case is determined, and test case optimization parameters are configured in combination with project requirements; and based on the parameter, calling a second preset test tool and a large language model to supplement the to-be-supplemented test case to obtain a complete test case. According to the scheme, the availability of the test case is improved through combination of multiple tools; workflows of test cases covering complex environments such as a function calling relation and a control flow structure are generated by utilizing the inference capability of a large language model, and the code coverage rate of the test cases is remarkably increased.

Description

Technical Field

[0001] The present application relates to the technical field of software testing optimization, and in particular to a test case generation method, apparatus, device and medium. Background Art

[0002] In the current software development process, code test coverage is an important indicator of code quality, and its level is directly affected by the quality of test cases. Traditional methods of generating test cases mainly rely on manual writing or the use of test case generation tools.

[0003] However, manually writing test cases is not only time-consuming but also prone to missing critical edge cases. While test case generation tools can quickly generate test cases, they often focus on the function dimension and ignore the requirements dimension, resulting in insufficient test coverage. This is particularly true for complex code structures, such as nested conditionals and loops, making it difficult to effectively improve coverage. Furthermore, the capabilities of individual tools are limited. These issues collectively hinder the improvement of code test coverage and impact the assurance of software quality.

[0004] In view of the above, how to solve the shortcomings of traditional test case generation methods in efficiency, coverage and complex logic processing, and the limited capabilities of a single tool, which makes it difficult to effectively improve code test coverage, is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0005] The purpose of this application is to provide a test case generation method, device, equipment and medium to solve the problem that traditional test case generation methods have shortcomings in efficiency, coverage and complex logic processing, and the limited capabilities of a single tool make it difficult to effectively improve code test coverage.

[0006] To solve the above technical problems, the present application provides a test case generation method, comprising:

[0007] Obtaining multiple initial test cases corresponding to the project code to be tested, and generating a test report corresponding to each of the initial test cases using a first preset testing tool;

[0008] According to each of the test reports, determine whether there are any test cases to be supplemented in each of the initial test cases whose corresponding code coverage indicators do not meet preset requirements;

[0009] If so, determining whether the code of the test case to be supplemented is not covered;

[0010] Configure test case optimization parameters based on project requirements and the code uncovered information;

[0011] Based on the test case optimization parameters, a second preset test tool and a large language model are called to supplement the test case to be supplemented to obtain a complete test case.

[0012] On the one hand, according to each of the test reports, it is determined whether there are any test cases to be supplemented in each of the initial test cases whose corresponding code coverage indicators do not meet the preset requirements, including:

[0013] Obtaining statement coverage and branch coverage in each of the test reports;

[0014] Determining whether there is the initial test case corresponding to the statement coverage and / or the branch coverage less than a threshold;

[0015] If so, the initial test case corresponding to the statement coverage and / or the branch coverage that is less than a threshold is determined as the test case to be supplemented.

[0016] On the other hand, determining code uncovered information of the test case to be supplemented includes:

[0017] Exporting the test case to be supplemented according to the format corresponding to the second preset test tool;

[0018] Determine uncovered statement information and / or uncovered branch information of the test case to be supplemented.

[0019] On the other hand, test case optimization parameters are configured according to project requirements and the code uncovered information, including:

[0020] Configure the test requirements and the contents of the function to be tested according to the project requirements;

[0021] Configure uncovered statement information according to the code uncovered information;

[0022] A first preset test case format is configured according to the first preset test tool.

[0023] On the other hand, based on the test case optimization parameters, calling a second preset test tool and a large language model to supplement the test case to be supplemented includes:

[0024] Supplementing the test case to be supplemented according to the second preset test tool and the test case optimization parameter to generate a first supplementary test case;

[0025] performing a test case format check and a function structure check on the first supplementary test case according to the test case optimization parameter and the large language model to generate a second supplementary test case;

[0026] According to the test case optimization parameters and the large language model, a requirement coverage check and a statement coverage check are performed on the second supplementary test case to generate the complete test case.

[0027] On the other hand, performing a test case format check and a function structure check on the first supplementary test case according to the test case optimization parameter and the large language model to generate a second supplementary test case, including:

[0028] Parsing the text content of the first supplementary test case and determining the corresponding test case format;

[0029] determining, according to the large language model, whether a test case format of the first supplementary test case is the same as the first preset test case format;

[0030] If it is confirmed that the test case format of the first supplementary test case is different from the first preset test case format, modifying the test case format of the first supplementary test case according to the large language model and the first preset test case format, and supplementing the first supplementary test case according to the large language model and the content of the function to be tested to generate the second supplementary test case;

[0031] If it is confirmed that the test case format of the first supplementary test case is the same as the first preset test case format, the first supplementary test case is directly supplemented according to the large language model and the content of the function to be tested to generate the second supplementary test case.

[0032] On the other hand, performing a requirement coverage check and a statement coverage check on the second supplementary test case according to the test case optimization parameters and the large language model to generate the complete test case, including:

[0033] Determining whether the test requirement is pre-configured;

[0034] If it is confirmed that the test requirement is not pre-configured, directly supplementing the second supplementary test case with statements based on the uncovered statement information and the large language model to generate the complete test case;

[0035] If the test requirement is confirmed to be pre-configured, determining whether the second supplementary test case meets the test requirement based on the large language model;

[0036] If it is confirmed that the second supplementary test case meets the test requirement, temporarily storing the second supplementary test case as a temporary result by variable assignment;

[0037] If it is determined that the second supplementary test case does not meet the test requirements, supplement the second supplementary test case according to the test requirements and the large language model, and temporarily store the second supplementary test case after the supplementation as a temporary result through variable assignment;

[0038] Determining whether the second supplementary test case in the temporary result covers all sentences according to the large language model;

[0039] If it is confirmed that the second supplementary test case in the provisional result covers all statements, determining the second supplementary test case in the provisional result as the complete test case;

[0040] If it is confirmed that the second supplementary test case in the temporary result does not cover all statements, the second supplementary test case is supplemented with statements based on the uncovered statement information and the large language model to generate the complete test case.

[0041] On the other hand, after obtaining the complete test case, it also includes:

[0042] Exporting the complete test case in a format corresponding to the first preset test tool;

[0043] Importing the complete test case into the first preset test tool and obtaining a test report for the complete test case;

[0044] According to the test report of the complete test case, determining whether the code coverage index of the complete test case does not meet the preset requirement;

[0045] If yes, the complete test case is determined as the test case to be supplemented, and the process proceeds to the step of determining code uncovered information of the test case to be supplemented;

[0046] If not, then end.

[0047] To solve the above technical problems, the present application also provides a test case generation device, comprising:

[0048] An acquisition module, configured to acquire a plurality of initial test cases corresponding to the project code to be tested, and generate a test report corresponding to each of the initial test cases using a first preset test tool;

[0049] A judgment module is used to judge whether there are any test cases to be supplemented in each of the initial test cases according to each of the test reports, and if so, trigger the determination module;

[0050] The determining module is used to determine the code uncovered information of the test case to be supplemented;

[0051] A configuration module, configured to configure test case optimization parameters according to project requirements and the code uncovered information;

[0052] The supplementing module is used to optimize parameters of the test case based on the test case, call a second preset test tool and a large language model to supplement the test case to be supplemented, so as to obtain a complete test case.

[0053] To solve the above technical problems, the present application also provides a test case generation device, comprising:

[0054] Memory for storing computer programs;

[0055] A processor is used to implement the steps of the above-mentioned test case generation method when executing the computer program.

[0056] In order to solve the above technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned test case generation method are implemented.

[0057] The test case generation method provided in the present application effectively reduces the time for manually writing test cases by obtaining multiple initial test cases corresponding to the project code to be tested, and generating test reports corresponding to each initial test case through a first preset test tool; when it is determined according to each test report that there are test cases to be supplemented whose corresponding code coverage indicators do not meet the preset requirements, the code non-coverage information of the test cases to be supplemented is determined, and the test case optimization parameters are configured in combination with the project requirements, so that based on the optimization parameters, the second preset test tool and the large language model are called to supplement the test cases to be supplemented, and a complete test case is obtained. This solution improves the availability of test cases by combining multiple test tools; at the same time, it uses the reasoning ability of the large language model to automatically generate a workflow for test cases that cover complex environments such as function call relationships and control flow structures, significantly improving the code coverage of test cases.

[0058] In addition, the present application also provides a test case generation device, equipment and medium, with the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0060] Figure 1 A flowchart of a test case generation method provided in an embodiment of the present application;

[0061] Figure 2 The overall flow chart for optimizing the test cases provided in the embodiments of the present application;

[0062] Figure 3 A schematic diagram of a test case generation device provided in an embodiment of the present application;

[0063] Figure 4 A structural diagram of a test case generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0065] The core of this application is to provide a test case generation method, device, equipment and medium to solve the problem that traditional test case generation methods have shortcomings in efficiency, coverage and complex logic processing, and the capabilities of a single tool are limited, making it difficult to effectively improve code test coverage.

[0066] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0067] In current software development, code test coverage is an important indicator of code quality, and its level is directly affected by the quality of test cases. Traditional methods for generating test cases rely primarily on manual writing or the use of test case generation tools. However, these methods face numerous challenges, as follows:

[0068] Manually writing test cases is not only time-consuming, but also prone to missing critical edge cases. Although test case generation tools can quickly generate test cases, they are often based on the function dimension and ignore the requirement dimension, resulting in insufficient test coverage. Especially for complex code structures, such as nested conditions and loops, traditional methods are difficult to effectively improve coverage. In addition, the capabilities of a single tool are also limited. For example, the SmartRocket tool can quickly generate test cases but has low coverage. Although the AITA tool can optimize use cases, it may generate useless content, thereby reducing coverage. Generating test cases using large language models faces high training and parameter adjustment costs. These problems jointly restrict the improvement of code test coverage and affect the assurance of software quality. In view of this, the present application provides a test case generation method that can effectively solve the above problems.

[0069] Figure 1 This is a flow chart of a test case generation method provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0070] S10: Acquire multiple initial test cases corresponding to the project code to be tested, and generate a test report corresponding to each initial test case through a first preset test tool.

[0071] First, the user inputs the project code to be tested, and the test tool generates multiple initial test cases corresponding to the project code to be tested. In this embodiment, there is no restriction on the test tool used to generate the initial test cases. For example, it can be SmartRocket, Automated Interface Test Assistant (AI Tessy Agent, AITA) and EvoSuite, etc., depending on the specific implementation situation. Taking into account the rapid generation of initial test cases, the SmartRocket tool can be used to generate multiple initial test cases in the specific implementation and output them in .yml format. It should be noted that SmartRocket is an efficient automated test case generation tool that quickly identifies key execution paths and conditions through static and dynamic code analysis, thereby generating a large number of test cases. It is particularly suitable for unit testing and regression testing, helping to verify the functionality of functions or methods and ensure the functional correctness after code modification. The advantage of SmartRocket lies in its rapid generation capability, which can significantly reduce the time of manually writing test cases.

[0072] Furthermore, multiple initial test cases are imported into the first preset test tool for testing, and a test report corresponding to each initial test case is generated by the first preset test tool. It should be noted that, in this embodiment, there is no restriction on the specific type of the first preset test tool, for example, it can be Tessy, Parasoft Jtest, EvoSuite, and Randoop, etc. Taking the use of the Tessy tool as an example of the first preset test tool, Tessy is an automated testing tool focused on embedded software, supporting unit testing and integration testing. It helps to achieve high coverage testing and improve code quality by automatically generating test cases and detailed coverage analysis. The main advantage of Tessy lies in its ability to handle the special needs of embedded systems and its high degree of automation. It can be integrated with continuous integration tools to reduce the workload of manual testing.

[0073] In addition, a test report generally includes the following: test pass rate, which shows the overall pass rate of test cases and reflects the overall success of the test; failed test cases, which lists all failed test cases, including the reasons for failure and related information, to help locate and resolve problems; code coverage, which provides code coverage analysis, such as statement coverage, branch coverage, and condition coverage, to demonstrate the comprehensiveness of code testing; uncovered code, which identifies uncovered code areas and guides subsequent test optimization; test steps, which records the execution steps of each test case to facilitate reproduction and debugging; expected results vs. actual results, which compares expected results with actual results to clarify the test verification criteria and results; execution time, which records the execution time of each test case and evaluates performance; resource consumption, which provides resource consumption data, such as memory and processor usage, to help identify performance bottlenecks; a test summary, which summarizes the test results and evaluates code quality and test effectiveness; and improvement suggestions, which provide suggestions for improving tests and code to guide subsequent development and testing work. Together, these contents constitute a test report, providing comprehensive test feedback to developers and testers. In this embodiment, there is no limitation on the specific content of the test report corresponding to the initial test case. It may include one or more of the above contents, and may also include other contents, depending on the specific implementation situation.

[0074] S11: Based on each test report, determine whether there are any test cases to be supplemented in each initial test case whose corresponding code coverage index does not meet the preset requirements. If yes, proceed to step S12; if not, end.

[0075] After obtaining the test reports, determine, based on each test report, whether there are any supplementary test cases in each initial test case whose corresponding code coverage indicators do not meet the preset requirements. It should be noted that code coverage indicators generally include statement coverage, branch coverage, condition coverage, path coverage, function coverage, and line coverage. In this embodiment, there is no restriction on the specific code coverage indicators involved, nor on the preset requirements, and it depends on the specific implementation situation.

[0076] If it is confirmed that any of the initial test cases have test cases that need to be supplemented and whose corresponding code coverage indicators do not meet the preset requirements, the subsequent steps are executed. If it is confirmed that no test cases need to be supplemented and whose corresponding code coverage indicators do not meet the preset requirements, the code coverage of all the initial test cases is considered to be qualified, and the initial test cases can be regarded as the final complete initial test cases, and this process ends.

[0077] S12: Determine code coverage information of the test cases to be supplemented.

[0078] When it is confirmed that there are test cases to be supplemented whose corresponding code coverage indicators do not meet the preset requirements in each initial test case, it is first necessary to determine the code coverage information of the test cases to be supplemented.

[0079] It should be noted that this embodiment does not limit the specific content of the code uncovered information. For example, it can be one or more of uncovered statement information, uncovered branch information, uncovered condition information, uncovered path information, uncovered function information, and uncovered line information, depending on the specific implementation situation.

[0080] S13: Configure test case optimization parameters based on project requirements and code coverage information.

[0081] Subsequently, the project requirements are determined, and the test case optimization parameters are configured according to the project requirements and the code uncovered information. It should be noted that the test case optimization parameters are configuration options or parameters used to adjust and improve the quality of the test cases. These parameters may include test case selection strategies, test requirements, function definitions, execution order, data generation algorithms, code uncovered information, etc. By reasonably setting and adjusting these parameters, the coverage rate, execution efficiency and defect detection ability of the test cases can be improved, thereby optimizing the testing process and improving software quality and testing efficiency. In this embodiment, there is no restriction on the content and specific configuration process of the test case optimization parameters, which shall be determined according to the specific implementation situation.

[0082] S14: Based on the test case optimization parameters, the second preset test tool and the large language model are called to supplement the test case to be supplemented to obtain a complete test case.

[0083] Finally, based on the configured test case optimization parameters, the second preset test tool and the large language model are called to supplement the test case to be supplemented to obtain a complete test case. It should be noted that in this embodiment, there is no restriction on the specific type of the second preset test tool, but it is necessary to ensure that it is different from the type of the first preset test tool. For example, the AITA tool can be used as the second preset test tool. Among them, the AITA tool can automatically generate detailed test specifications, including test case templates, test scenarios, test data and expected results, through intelligent analysis of interface definitions and requirement documents. It can significantly improve test efficiency, ensure the correctness of interface functions, and provide detailed test reports and coverage analysis to help development and testing teams comprehensively verify interface quality and optimize testing processes.

[0084] It should be noted that the Large Language Model (LLM) is a deep learning-based neural network model, trained on large-scale text data and possessing powerful language understanding and generation capabilities. LLM has demonstrated outstanding performance in applications such as text summarization, machine translation, and dialogue systems. When it comes to test case generation, LLM automatically generates comprehensive and accurate test cases by analyzing code logic and requirement descriptions, including normal, abnormal, and edge cases, significantly reducing the workload of manual coding. Its ability to understand complex logic and potential paths enables the generated test cases to cover a variety of execution paths, improving test comprehensiveness and effectiveness. LLM also generates reasonable test data based on parameter types and ranges, ensuring test case diversity and representativeness. Furthermore, the test cases generated by LLM are highly readable and maintainable, making them easy to understand, execute, and subsequently maintain and update, thereby improving testing efficiency and quality. Consequently, in this embodiment, the complete test cases generated by invoking the second preset test tool and the large language model to supplement the supplementary test cases have superior code coverage and quality.

[0085] In this embodiment, there is no restriction on the specific type of the selected large language model, and there is no restriction on the specific process of calling the second preset test tool and the large language model to supplement the supplementary test case, which depends on the specific implementation situation.

[0086] In this embodiment, by obtaining multiple initial test cases corresponding to the project code to be tested, and generating test reports corresponding to each initial test case through a first preset test tool, the time for manually writing test cases is effectively reduced; when it is determined according to each test report that there are test cases to be supplemented whose corresponding code coverage indicators do not meet the preset requirements, the code uncovered information of the test cases to be supplemented is determined, and the test case optimization parameters are configured in combination with the project requirements, so that based on the optimization parameters, the second preset test tool and the large language model are called to supplement the test cases to be supplemented, and a complete test case is obtained. This solution improves the availability of test cases by combining multiple test tools; at the same time, by utilizing the reasoning ability of the large language model, a workflow for test cases covering complex environments such as function call relationships and control flow structures is automatically generated, significantly improving the code coverage of test cases.

[0087] Based on the above embodiment, in some embodiments, judging, based on each test report, whether there are any test cases to be supplemented whose corresponding code coverage indicators do not meet preset requirements in each initial test case includes:

[0088] S111: Obtain statement coverage and branch coverage in each test report.

[0089] S112: Determine whether there is an initial test case whose corresponding statement coverage and / or branch coverage is less than a threshold; if so, proceed to step S113; if not, end.

[0090] S113: Determine the initial test cases whose corresponding statement coverage and / or branch coverage is less than a threshold as test cases to be supplemented.

[0091] In order to determine whether there are test cases to be supplemented, in this embodiment, the statement coverage and branch coverage in each test report are specifically obtained. It is worth noting that statement coverage (C0) and branch coverage (C1) are important indicators for measuring the comprehensiveness of code testing. Statement coverage ensures that each code statement is executed at least once during the test process to discover potential syntax or execution errors. Branch coverage is more stringent and requires that each conditional branch (such as if and else) be executed at least once during the test to catch possible logical errors. For example, for a conditional statement, not only the if part but also the else part must be executed to ensure that all possible logical paths are tested.

[0092] Further, it is determined whether there is an initial test case whose corresponding statement coverage and / or branch coverage is less than the threshold. It should be noted that in this embodiment, there is no restriction on the threshold value, for example, it can be 100 , that is, to determine whether there is a corresponding C0 and / or C1 less than 100 Initial test case.

[0093] If it is confirmed that there is no initial test case with a corresponding statement coverage and / or branch coverage less than the threshold, the initial test case can be directly used as a complete test case, and the judgment process ends. If it is confirmed that there is an initial test case with a corresponding statement coverage and / or branch coverage less than the threshold, the initial test case with a corresponding statement coverage and / or branch coverage less than the threshold is determined as a test case to be supplemented, so as to facilitate subsequent supplementation.

[0094] In this way, the test cases to be supplemented are determined, so that the test cases to be supplemented can be supplemented and improved, thereby improving software quality and testing efficiency.

[0095] Based on the above embodiment, in some embodiments, determining code uncovered information of the test case to be supplemented includes:

[0096] S121: Export the test case to be supplemented according to the format corresponding to the second preset test tool.

[0097] S122: Determine uncovered statement information and / or uncovered branch information of the test case to be supplemented.

[0098] After determining the test cases to be supplemented, in order to better supplement the test cases to be supplemented, it is first necessary to export the test cases to be supplemented in the format corresponding to the second preset test tool, so that the exported test cases to be supplemented can be directly imported into the second preset test tool for supplementation. It should be noted that in this embodiment, there is no restriction on the format corresponding to the second preset test tool, which depends on the type of the second preset test tool specifically selected. For example, when the second preset test tool selects the AITA tool, the test cases to be supplemented can be exported in the .script format. Subsequently, according to the corresponding test report content, the uncovered statement information and / or uncovered branch information of the test cases to be supplemented are determined, and the insufficient statement coverage and branch coverage are recorded for subsequent supplementation.

[0099] Based on the above embodiments, in some embodiments, configuring test case optimization parameters according to project requirements and code uncovered information includes:

[0100] S131: Configure test requirements and the contents of the function to be tested according to project requirements.

[0101] S132: Configure uncovered statement information according to the code uncovered information.

[0102] S133: Configuring a first preset test case format according to a first preset test tool.

[0103] In this embodiment, test case optimization parameters are specifically configured based on the Dify platform window. The Dify platform provides a rich set of functional modules and flexible configuration options, enabling users to quickly design and implement workflows based on specific business needs. Through a graphical interface, users can intuitively drag and connect different task nodes to define data flows and business logic. The Dify platform supports multiple triggering methods, including timed triggering, event triggering, and application programming interface calls, ensuring that workflows can respond to various business scenarios in a timely manner. During workflow execution, the Dify platform provides real-time monitoring and logging functions to help users identify and resolve problems in a timely manner.

[0104] Therefore, in implementation, specifically based on the Dify platform window, the test requirements and the content of the function to be tested required for the test case optimization workflow under the Dify platform are configured and filled in according to project requirements. The code uncovered information obtained in the above embodiment is configured and filled in with uncovered statement information. Finally, based on the first preset test tool used, the test case format specification is configured to the corresponding first preset test case format. Taking the Tessy tool as an example, it must be contained by $testobject, at least one $testcase and at least one $teststep must exist, and the testcase must have $name and $uuid.

[0105] In summary, the configuration of test case optimization parameters is implemented based on the Dify platform window, so that the test case optimization workflow based on the Dify platform can be used to optimize the supplementary test cases.

[0106] Based on the above embodiments, in some embodiments, based on the test case optimization parameters, calling the second preset test tool and the large language model to supplement the test case to be supplemented includes:

[0107] S141: Supplement the test case to be supplemented according to the second preset test tool and test case optimization parameters to generate a first supplementary test case.

[0108] S142: Perform a test case format check and a function structure check on the first supplementary test case according to the test case optimization parameters and the large language model to generate a second supplementary test case.

[0109] S143: Perform a requirement coverage check and a statement coverage check on the second supplementary test case according to the test case optimization parameters and the large language model to generate a complete test case.

[0110] Specifically, the test case to be supplemented is first imported into the second preset test tool, and the test case to be supplemented is preliminarily supplemented based on the uncovered statement information in the test case optimization parameters to generate a first supplementary test case. For example, the GenerateTestSpecification interface of the AITA tool is called to preliminarily supplement the test case to be supplemented.

[0111] Furthermore, based on the test case optimization parameters and the large language model, the first supplementary test case is subjected to a test case format check and a function structure check to generate a second supplementary test case. Furthermore, based on the test case optimization parameters and the large language model, the second supplementary test case is subjected to a requirement coverage check and a statement coverage check to generate a complete test case. The specific processes of the test case format check, function structure check, requirement coverage check, and statement coverage check are described in detail below with reference to specific embodiments:

[0112] In some embodiments, performing a test case format check and a function structure check on the first supplementary test case based on the test case optimization parameters and the large language model to generate a second supplementary test case includes:

[0113] S151: Parse the text content of the first supplementary test case and determine the corresponding test case format.

[0114] S152: Determine whether the test case format of the first supplementary test case is the same as the first preset test case format based on the large language model; if not, proceed to step S153; if so, proceed to step S154.

[0115] S153: Modify the test case format of the first supplementary test case according to the large language model and the first preset test case format, and supplement the first supplementary test case according to the large language model and the content of the function to be tested to generate a second supplementary test case.

[0116] S154: Directly supplement the first supplementary test case based on the large language model and the content of the function to be tested to generate a second supplementary test case.

[0117] Specifically, the text content of the first supplementary test case is first parsed and the corresponding test case format is determined. Then, based on the large language model, it is determined whether the test case format of the first supplementary test case is the same as the first preset test case format.

[0118] If it is confirmed that the test case format of the first supplementary test case is different from the first preset test case format, it is considered that the test case format of the current first supplementary test case is incorrect and needs to be modified; specifically, the test case format of the first supplementary test case is modified to the first preset test case format based on the large language model. After the modification is completed, it can be further determined whether the modification is successful; if the modification is successful, the subsequent steps are performed; if the modification fails, the test case generation failure information is output, prompting the user to make manual adjustments, and the entire process ends. After the test case format modification is completed and confirmed to be successful, the first supplementary test case is supplemented according to the large language model and the content of the function to be tested to generate a second supplementary test case. If it is confirmed that the test case format of the first supplementary test case is the same as the first preset test case format, the first supplementary test case is directly supplemented according to the large language model and the content of the function to be tested to generate a second supplementary test case.

[0119] It should also be noted that after generating the second supplementary test case, considering that the content generated in the call_trace part of the test case generated by the AITA tool may be unstable, occasional errors will occur when it is subsequently imported into the Tessy tool. The large language model can also be used to shield the call_trace part of the optimized test case to ensure the stability of the test case.

[0120] In some embodiments, based on the test case optimization parameters and the large language model, a requirement coverage check and a statement coverage check are performed on the second supplementary test case to generate a complete test case, including:

[0121] S161: Determine whether the test requirements are pre-configured; if not, proceed to step S166; if yes, proceed to step S162.

[0122] S162: Determine whether the second supplementary test case meets the test requirements based on the large language model; if so, proceed to step S163; if not, proceed to step S164.

[0123] S163: Temporarily save the second supplementary test case as a temporary result by assigning a value to a variable.

[0124] S164: Supplement the second supplementary test case according to the test requirements and the large language model, and temporarily store the supplemented second supplementary test case as a temporary result through variable assignment.

[0125] S165: Determine whether the second supplementary test case in the temporary result covers all sentences based on the large language model; if so, proceed to step S167; if not, proceed to step S166.

[0126] S166: Based on the uncovered sentence information and the large language model, sentences are supplemented for the second supplementary test case to generate a complete test case.

[0127] S167: Determine the second supplementary test case in the temporary result as a complete test case.

[0128] Specifically, the system first determines whether the user has pre-configured test requirements in the Dify platform window. If not, the system then directly adds statements to the second supplementary test case based on the uncovered statement information and the large language model to generate a complete test case. If the user has pre-configured test requirements, the system then uses the large language model to determine whether the second supplementary test case meets the configured test requirements.

[0129] If it is confirmed that the second supplementary test case meets the pre-configured test requirements, the second supplementary test case is temporarily stored as a temporary result temp_result through variable assignment. If it is confirmed that the second supplementary test case does not meet the pre-configured test requirements, the second supplementary test case is supplemented according to the test requirements and the large language model, and the second supplementary test case with the supplemented requirements is temporarily stored as a temporary result temp_result through variable assignment.

[0130] Then, determine whether the second supplementary test case in the temporary result temp_result covers all statements based on the large language model. If it is confirmed that the second supplementary test case in the temporary result covers all statements, the second supplementary test case in the temporary result is determined to be a complete test case, and a prompt message indicating that the test case has been optimized based on the function, test requirements and coverage report is output to prompt the user to keep going. If it is confirmed that the second supplementary test case in the temporary result does not cover all statements, the second supplementary test case is supplemented with statements based on the uncovered statement information and the large language model to generate a complete test case, and a prompt message indicating that the test case has been optimized based on the function, test requirements and coverage report is output to prompt the user to keep going.

[0131] In summary, in this embodiment, a workflow is created by using the Dify platform to generate test cases based on a large language model that covers multiple logical paths and boundary conditions. Code features are then extracted through grammatical and semantic analysis, and the reasoning ability of the large language model is utilized to automatically generate test cases that cover complex environments such as function call relationships and control flow structures, effectively improving the coverage of test cases.

[0132] Figure 2 The test case optimization overall flow chart provided in the embodiment of this application. Figure 2 As shown, based on the above embodiment, in some embodiments, after obtaining a complete test case, the following is further included:

[0133] S171: Export the complete test case in a format corresponding to the first preset test tool;

[0134] S172: Import the complete test case into the first preset test tool and obtain a test report for the complete test case;

[0135] S173: According to the test report of the complete test case, determine whether the code coverage index of the complete test case does not meet the preset requirements; if so, proceed to step S174; if not, end.

[0136] S174: Determine the complete test case as a test case to be supplemented, and proceed to step S12.

[0137] In order to ensure that the code coverage of the test case reaches 100%, after obtaining the complete test case after supplementation, the complete test case needs to be exported in the format corresponding to the first preset test tool. For example, when the first preset test tool is the Tessy tool, the complete test case can be exported in the .yml format, .script format or .txt format supported by the Tessy tool. On this basis, considering that the AITA tool was used to supplement the test case before, and the AITA tool currently only supports the import of .script format data, in order to facilitate the subsequent iterative debugging of the test case, the input and output formats of the test case should be consistent as much as possible. Therefore, in the specific implementation, you can choose to export the complete test case in .script format.

[0138] Furthermore, the complete test case is imported into the first preset test tool, and a test report for the complete test case is obtained. Subsequently, based on the test report for the complete test case, it is determined whether the code coverage indicator of the complete test case does not meet the preset requirement. It is understood that the code coverage indicator and the preset requirement determined here should be the same as the code coverage indicator and the preset requirement mentioned in the above embodiment.

[0139] If the code coverage of the complete test case is confirmed to meet the preset requirements, the entire process ends. If the code coverage of the complete test case is confirmed to not meet the preset requirements, the complete test case needs to be iteratively supplemented, and the complete test case is identified as a test case to be supplemented. The step of determining the code coverage information of the test case to be supplemented is carried out until the preset requirements are met.

[0140] In this way, the parameters of the large language model in the workflow are fine-tuned based on the code coverage indicators of the complete test cases, further improving the quality of the test cases and forming a closed-loop optimization mechanism.

[0141] In the above embodiments, the test case generation method is described in detail, and the present application also provides corresponding embodiments of the test case generation device.

[0142] Figure 3 This is a schematic diagram of a test case generation device provided in an embodiment of the present application. Figure 3 As shown, the device includes:

[0143] The acquisition module 10 is used to acquire a plurality of initial test cases corresponding to the project code to be tested, and generate a test report corresponding to each initial test case through a first preset test tool.

[0144] The judgment module 11 is used to judge whether there are any test cases to be supplemented in each initial test case according to each test report, and the corresponding code coverage index does not meet the preset requirements; if so, the determination module 12 is triggered.

[0145] The determination module 12 is used to determine the code coverage information of the test case to be supplemented.

[0146] The configuration module 13 is used to configure test case optimization parameters according to project requirements and code coverage information.

[0147] The supplementing module 14 is configured to optimize parameters based on the test case, and call a second preset test tool and a large language model to supplement the test case to be supplemented, so as to obtain a complete test case.

[0148] In some embodiments, the determination module 11 includes:

[0149] The first acquisition submodule is used to obtain the statement coverage and branch coverage in each test report;

[0150] The first judgment submodule is used to determine whether there is an initial test case whose corresponding statement coverage and / or branch coverage is less than a threshold; if so, the first determination submodule is used.

[0151] The first determining submodule is configured to determine initial test cases whose corresponding statement coverage and / or branch coverage is less than a threshold as test cases to be supplemented.

[0152] In some embodiments, the determination module 12 includes:

[0153] A first export submodule is used to export the test cases to be supplemented according to the format corresponding to the second preset test tool;

[0154] The second determining submodule is used to determine uncovered statement information and / or uncovered branch information of the test case to be supplemented.

[0155] In some embodiments, the configuration module 13 includes:

[0156] The first configuration submodule is used to configure the test requirements and the contents of the function to be tested according to the project requirements;

[0157] The second configuration submodule is used to configure uncovered statement information according to the code uncovered information;

[0158] The third configuration submodule is used to configure the first preset test case format according to the first preset test tool.

[0159] In some embodiments, the supplement module 14 includes:

[0160] A first supplementing submodule, configured to supplement the test case to be supplemented according to the second preset test tool and test case optimization parameters to generate a first supplementing test case;

[0161] a second supplementary submodule, configured to optimize parameters and the large language model according to the test case, perform a test case format check and a function structure check on the first supplementary test case, and generate a second supplementary test case;

[0162] The third supplementary submodule is used to optimize parameters and the large language model according to the test case, and perform requirement coverage check and statement coverage check on the second supplementary test case to generate a complete test case.

[0163] In some embodiments, the second supplementary submodule includes:

[0164] A parsing submodule, configured to parse the text content of the first supplementary test case and determine the corresponding test case format;

[0165] a second judgment submodule, configured to determine, based on the large language model, whether the test case format of the first supplementary test case is the same as the first preset test case format; if not, triggering the first modification submodule; if so, triggering the fourth supplementation submodule;

[0166] a first modifying submodule, configured to modify the test case format of the first supplementary test case according to the large language model and the first preset test case format, and supplement the first supplementary test case according to the large language model and the content of the function to be tested to generate a second supplementary test case;

[0167] The fourth supplementing submodule is used to supplement the first supplementary test case directly according to the large language model and the content of the function to be tested, so as to generate a second supplementary test case.

[0168] In some embodiments, the third supplementary submodule includes:

[0169] The third judgment submodule is used to judge whether the test requirements are pre-configured; if not, the generation submodule is triggered; if so, the fourth judgment submodule is triggered;

[0170] The fourth judgment submodule is used to judge whether the second supplementary test case meets the test requirements according to the large language model; if so, the first temporary storage submodule is triggered; if not, the fifth supplementary submodule is triggered;

[0171] A first temporary storage submodule, configured to temporarily store the second supplementary test case as a temporary result by assigning a value to a variable;

[0172] A fifth supplementing submodule is configured to supplement the second supplementary test case according to the test requirements and the large language model, and temporarily store the supplemented second supplementary test case as a temporary result by assigning a value to a variable;

[0173] A fifth judgment submodule is configured to judge whether the second supplementary test case in the temporary result covers all the sentences based on the large language model; if so, trigger the third determination submodule; if not, trigger the generation submodule;

[0174] A third determining submodule is configured to determine the second supplementary test case in the temporary result as a complete test case;

[0175] The generation submodule is used to supplement the second supplementary test case with sentences based on the uncovered sentence information and the large language model to generate a complete test case.

[0176] In some embodiments, further comprising:

[0177] The second export submodule is used to export the complete test case in a format corresponding to the first preset test tool;

[0178] A first import submodule is used to import the complete test case into the first preset test tool and obtain a test report for the complete test case;

[0179] The sixth judgment submodule is used to judge whether the code coverage index of the complete test case does not meet the preset requirements based on the test report of the complete test case; if so, the complete test case is determined as a test case to be supplemented, triggering the determination module 12; if not, it ends.

[0180] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, and they will not be repeated here.

[0181] Figure 4 This is a structural diagram of a test case generation device provided in an embodiment of the present application. Figure 4 As shown, the test case generation device includes:

[0182] Memory 20, for storing computer programs;

[0183] The processor 21 is configured to implement the steps of the test case generation method mentioned in the above embodiment when executing a computer program.

[0184] The test case generation device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0185] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented in at least one hardware form: a digital signal processor (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is responsible for processing computing operations related to machine learning.

[0186] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the test case generation method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to data involved in the test case generation method.

[0187] In some embodiments, the test case generating device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0188] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation to the test case generating device, and may include more or fewer components than shown in the figure.

[0189] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiment.

[0190] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0191] The above is a detailed introduction to a test case generation method, device, equipment and medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of this application.

[0192] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A test case generation method, characterized in that: include: Obtaining multiple initial test cases corresponding to the project code to be tested, and generating a test report corresponding to each of the initial test cases using a first preset testing tool; According to each of the test reports, determine whether there are any test cases to be supplemented in each of the initial test cases whose corresponding code coverage indicators do not meet preset requirements; If so, determining whether the code of the test case to be supplemented is not covered; Configure test case optimization parameters based on project requirements and the code uncovered information; Based on the test case optimization parameters, a second preset test tool and a large language model are called to supplement the test case to be supplemented to obtain a complete test case.

2. The test case generation method according to claim 1, characterized in that: According to each of the test reports, it is determined whether there are any test cases to be supplemented in each of the initial test cases whose corresponding code coverage indicators do not meet the preset requirements, including: Obtaining statement coverage and branch coverage in each of the test reports; Determining whether there is the initial test case corresponding to the statement coverage and / or the branch coverage less than a threshold; If so, the initial test case corresponding to the statement coverage and / or the branch coverage that is less than a threshold is determined as the test case to be supplemented.

3. The test case generation method according to claim 2, characterized in that: Determine the code coverage information of the test case to be supplemented, including: Exporting the test case to be supplemented according to the format corresponding to the second preset test tool; Determine uncovered statement information and / or uncovered branch information of the test case to be supplemented.

4. The test case generation method according to claim 1, wherein: Configure test case optimization parameters based on project requirements and the code uncovered information, including: Configure the test requirements and the contents of the function to be tested according to the project requirements; Configure uncovered statement information according to the code uncovered information; A first preset test case format is configured according to the first preset test tool.

5. The test case generation method according to claim 4, characterized in that: Based on the test case optimization parameters, calling a second preset test tool and a large language model to supplement the test case to be supplemented includes: Supplementing the test case to be supplemented according to the second preset test tool and the test case optimization parameter to generate a first supplementary test case; performing a test case format check and a function structure check on the first supplementary test case according to the test case optimization parameter and the large language model to generate a second supplementary test case; According to the test case optimization parameters and the large language model, a requirement coverage check and a statement coverage check are performed on the second supplementary test case to generate the complete test case.

6. The test case generation method according to claim 5, characterized in that: Performing a test case format check and a function structure check on the first supplementary test case according to the test case optimization parameter and the large language model to generate a second supplementary test case, including: Parsing the text content of the first supplementary test case and determining the corresponding test case format; determining, according to the large language model, whether a test case format of the first supplementary test case is the same as the first preset test case format; If it is confirmed that the test case format of the first supplementary test case is different from the first preset test case format, modifying the test case format of the first supplementary test case according to the large language model and the first preset test case format, and supplementing the first supplementary test case according to the large language model and the content of the function to be tested to generate the second supplementary test case; If it is confirmed that the test case format of the first supplementary test case is the same as the first preset test case format, the first supplementary test case is directly supplemented according to the large language model and the content of the function to be tested to generate the second supplementary test case.

7. The test case generation method according to claim 5, characterized in that: Performing a requirement coverage check and a statement coverage check on the second supplementary test case according to the test case optimization parameter and the large language model to generate the complete test case, including: Determining whether the test requirement is pre-configured; If it is confirmed that the test requirement is not pre-configured, directly supplementing the second supplementary test case with statements based on the uncovered statement information and the large language model to generate the complete test case; If the test requirement is confirmed to be pre-configured, determining whether the second supplementary test case meets the test requirement based on the large language model; If it is confirmed that the second supplementary test case meets the test requirement, temporarily storing the second supplementary test case as a temporary result by variable assignment; If it is determined that the second supplementary test case does not meet the test requirements, supplement the second supplementary test case according to the test requirements and the large language model, and temporarily store the second supplementary test case after the supplementation as a temporary result through variable assignment; Determining whether the second supplementary test case in the temporary result covers all sentences according to the large language model; If it is confirmed that the second supplementary test case in the provisional result covers all statements, determining the second supplementary test case in the provisional result as the complete test case; If it is confirmed that the second supplementary test case in the temporary result does not cover all statements, the second supplementary test case is supplemented with statements based on the uncovered statement information and the large language model to generate the complete test case.

8. The test case generation method according to any one of claims 1 to 7, characterized in that: After obtaining the complete test case, it also includes: Exporting the complete test case in a format corresponding to the first preset test tool; Importing the complete test case into the first preset test tool and obtaining a test report for the complete test case; According to the test report of the complete test case, determining whether the code coverage index of the complete test case does not meet the preset requirement; If yes, the complete test case is determined as the test case to be supplemented, and the process proceeds to the step of determining code uncovered information of the test case to be supplemented; If not, then end.

9. A test case generating device, characterized in that: include: An acquisition module, configured to acquire a plurality of initial test cases corresponding to the project code to be tested, and generate a test report corresponding to each of the initial test cases using a first preset test tool; A judgment module is used to judge whether there are any test cases to be supplemented in each of the initial test cases according to each of the test reports, and if so, trigger a determination module; The determining module is used to determine the code uncovered information of the test case to be supplemented; A configuration module, configured to configure test case optimization parameters according to project requirements and the code uncovered information; The supplementing module is used to optimize parameters of the test case based on the test case, call a second preset test tool and a large language model to supplement the test case to be supplemented, so as to obtain a complete test case.

10. A test case generation device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the test case generation method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the test case generation method according to any one of claims 1 to 8 are implemented.

Images