Test case generation method and system based on large model and retrieval enhancement generation

Abstract

The invention provides a test case generation method and system based on a large model and retrieval enhancement generation, and the method comprises the steps: obtaining a software description document, carrying out text fragment segmentation and vectorization, and constructing a software knowledge base; obtaining and vectorizing a demand document, and recalling a content optimization demand document in the reference software knowledge base based on the retrieval enhancement generation framework; constructing a first Prompt instruction, and splitting the optimized demand document into demand fragments of different test points by using a large language model; and finally, constructing a second Prompt instruction for the demand fragment of each test point, generating a test case by using a large language model, and establishing a mapping relationship between each test point and the test case for linkage output. According to the method, the understanding ability of the large model on the demand document is effectively enhanced; meanwhile, the requirements are decomposed into test points, mapping is carried out on the test points and related test cases, and completeness and accuracy of the requirements and the test cases can be conveniently sorted.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and test case generation, and more specifically, to a test case generation method and system based on large model and retrieval enhanced generation. Background Art

[0002] A test case is a set of test steps and input data written during the software testing process to verify whether the software system meets the expected functional and performance requirements. Test cases usually include information such as test purpose, test conditions, test steps, and expected results, and are intended to verify the correctness, stability, reliability, and security of the software system. By executing test cases, defects and problems in the software system can be discovered, and repairs and improvements can be made in a timely manner to improve software quality and user satisfaction.

[0003] Traditionally, test case writing is mainly undertaken by testers, who need to study the requirements documents in depth, accurately grasp the requirements and expected behaviors of each functional module, and have multiple discussions with product managers and developers to ensure that the details and boundary conditions of the requirements are clear. However, this traditional manual writing method is not only time-consuming and labor-intensive, but also difficult to achieve uniform quality.

[0004] In recent years, with the increasing maturity of Large Language Model (LLM) technology, more and more products have begun to use the reasoning ability of large models and their own knowledge reserves to automatically understand requirement documents and generate test cases or assist in generating intermediate products (such as dividing test modules, refining test points, etc.). Compared with traditional methods, this automated testing solution has significantly improved efficiency and also helps to ensure the standardization of use cases.

[0005] For example, the prior art discloses a test case generation method, device and equipment, the method comprising: planning a generation strategy corresponding to each element contained in the test case to be generated; generating prompt instructions corresponding to each element according to the generation strategy and generation requirements corresponding to each element, inputting the prompt instructions corresponding to each element into a pre-trained vertical domain large language model to generate content corresponding to each element; the vertical domain large language model is trained according to the vertical domain knowledge corpus and the use case case library corresponding to the test case; filling in the content corresponding to each element according to the use case template to generate a test case; however, most of the automatic test case writing solutions based on large models, including the prior art, still have limitations:

[0006] 1) Large models are mostly trained based on public data, and lack understanding of terminology and processes in specific fields;

[0007] 2) Large models generally suffer from hallucinations. They pretend to understand things they don’t understand and output many invalid use cases.

[0008] 3) The formats of actual requirement documents vary greatly, and there is a lot of redundant and invalid information, which affects the accuracy of use case generation;

[0009] 4) The use cases generated by the large model include invalid use cases and valid use cases, and it is difficult for existing technologies to select valid use cases from them;

[0010] 5) The requirement document is directly input into the big model. A complete requirement document directly generates multiple test cases at one time. There is no mapping between the requirement decomposition test points and each test case, which makes it inconvenient for testers to perform requirement or case completeness analysis. Summary of the invention

[0011] In order to overcome the defects of low efficiency and accuracy of test case generation using big models in the above-mentioned prior art, as well as the lack of feedback mechanism and lack of association mapping between requirements and use cases, the present invention provides a test case generation method and system based on big models and retrieval-augmented generation, which can quickly generate test cases based on requirement documents, and combine with retrieval-augmented generation (RAG) technology to effectively enhance the big model's ability to understand requirement documents; at the same time, the requirements are decomposed into test points and mapped with related test cases, so that the completeness and accuracy of requirements and test cases can be conveniently sorted out.

[0012] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0013] A test case generation method based on a large model and retrieval enhanced generation, comprising:

[0014] S1: Obtain software documentation and perform text segmentation and vectorization, and use all vectorized data to build a software knowledge base;

[0015] S2: Obtain the requirement document and vectorize it, and recall and reference the content in the software knowledge base based on the retrieval enhancement generation framework to obtain the optimized requirement document;

[0016] S3: construct a first Prompt instruction, input the optimized requirement document into at least one large language model, and use the first Prompt instruction as a prompt word to split the optimized requirement document into requirement fragments of several test points;

[0017] S4: Construct a second Prompt instruction for each test point requirement fragment, use each of the second Prompt instructions as a prompt word of at least one large language model, generate test cases corresponding to each test point, establish a mapping relationship between the test points and the test cases for linkage output.

[0018] Preferably, the method further comprises:

[0019] S5: Obtaining user feedback information on the generated test case, and adjusting the second Prompt instruction according to the feedback information;

[0020] Repeat step S4, regenerate a new test case according to the new second Prompt instruction, establish a new mapping relationship between the test point and the new test case for linkage output.

[0021] Preferably, the format of the linkage output includes: any one of: BDD format, manual test case format, Excel format and Xmind mind map format.

[0022] Preferably, in step S5, the feedback information includes any one or more of the following:

[0023] The user's check result on whether the optimal test case has ambiguity or vague description;

[0024] User feedback on the compatibility of the best test cases running on different platforms;

[0025] The user adds or deletes software test points;

[0026] User requirements for boundary test cases;

[0027] User demand for exceptional test cases.

[0028] Preferably, in step S1, the software description document includes: a product introduction of the software, background knowledge and a functional description of each test point.

[0029] Preferably, in step S2, the format of the demand document includes any one of doc format, pdf format, txt format and markdown format.

[0030] Preferably, in step S2, the requirement document includes: business rules for user operations and test points.

[0031] Preferably, in step S3, the large language model includes any one or more of GPT-4o, Claude-3.5, Gemini-1.5, GLM-4, QWen, ERNIE and Hunyuan.

[0032] The present invention also provides a test case generation system based on a large model and retrieval enhanced generation, which applies the above-mentioned test case generation method based on a large model and retrieval enhanced generation, including:

[0033] Knowledge base construction unit: used to obtain software description documents and perform text segmentation and vectorization, and use all vectorized data to build a software knowledge base;

[0034] Requirement document optimization unit: used to obtain and vectorize the requirement document, recall and reference the content in the software knowledge base based on the retrieval enhancement generation framework, and obtain the optimized requirement document;

[0035] A requirement document splitting unit is used to construct a first Prompt instruction, input the optimized requirement document into at least one large language model, and use the first Prompt instruction as a prompt word to split the optimized requirement document into requirement fragments of a plurality of test points;

[0036] Test case generation unit: used to construct a second Prompt instruction for the requirement fragment of each test point, use each of the second Prompt instructions as the prompt word of at least one large language model, generate test cases corresponding to each test point, establish a mapping relationship between the test point and the test case for linkage output.

[0037] Preferably, the system further comprises:

[0038] A test case updating unit: used to obtain user feedback information on the generated test case, and adjust the second Prompt instruction according to the feedback information;

[0039] The test case generation unit is executed repeatedly, a new test case is regenerated according to the new second Prompt instruction, and a new mapping relationship is established between the test point and the new test case for linkage output.

[0040] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0041] The present invention provides a test case generation method based on a large model and retrieval enhanced generation. First, a software description document is obtained and text segmentation and vectorization are performed, and a software knowledge base is constructed using all vectorized data; a requirement document is obtained and vectorized, and the content in the software knowledge base is recalled and referenced based on a retrieval enhanced generation framework to obtain an optimized requirement document; a first Prompt instruction is constructed, and the optimized requirement document is input into at least one large language model, and the first Prompt instruction is used as a prompt word to split the optimized requirement document into requirement segments of a plurality of test points; a second Prompt instruction is constructed for the requirement segment of each test point, and each second Prompt instruction is used as a prompt word of at least one large language model, to generate test cases corresponding to each test point, and a mapping relationship is established between the test point and the test case for linkage output;

[0042] The present invention has the following beneficial effects:

[0043] 1) The present invention can automatically split the input requirement document into test points, automatically generate structured test cases, and map the test cases to the requirement test points; the present invention not only improves the efficiency of test case writing, significantly saves the time of testers writing test cases from 0 to 1, but also ensures the uniformity and standardization of the output format;

[0044] 2) The present invention uses RAG technology to dynamically recall the background knowledge and function-related key points, detailed descriptions and other contents in the knowledge base of relevant business software products, so that the large language model can fully understand the specific function key points of the relevant software products and the specific meanings behind the various proprietary function nouns of the software products from the simple requirements document, and finally achieve the effect of expanding the content of the requirements document, thereby ensuring that the generated test cases have high relevance and accuracy;

[0045] 3) The present invention reduces the probability of hallucinations through cross-validation of multiple large language models, analyzes and selects the most reasonable demand test points, and then maps them to relevant test cases, which can facilitate the sorting out of the completeness and accuracy of the requirements and test cases;

[0046] 4) The present invention designs a complete processing link, which can generate test cases through multiple rounds of dialogue to continuously improve the test cases; based on the existing generated cases, it can check for omissions and fill in the gaps according to the needs of testers, emphasize the supplementation of test cases in a certain module or field, expand abnormal boundary test cases, supplement more test cases, etc., so that the large language model has a clearer purpose when generating test cases, and achieves higher accuracy and recall. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1This is a flow chart of a test case generation method based on a large model and retrieval enhanced generation provided in Example 1.

[0048] Figure 2 A flowchart for constructing the software knowledge base provided in Example 2.

[0049] Figure 3 Generate a flow chart for the Prompt instruction provided in Example 2.

[0050] Figure 4 The flowchart of generating test cases by using multiple large models structured as provided in Example 2.

[0051] Figure 5 Schematic diagram of test cases generated by the reference knowledge base provided in Example 2.

[0052] Figure 6 Schematic diagram of test cases generated by the unreferenced knowledge base provided in Example 2.

[0053] Figure 7 A flowchart is generated for the multiple rounds of test case iterations provided in Example 2.

[0054] Figure 8 This is a structure diagram of a test case generation system based on a large model and retrieval enhancement generation provided in Example 3.

[0055] Fig. 9 This is an architecture diagram of a test case generation platform based on a large model and retrieval enhancement generation provided in Example 4. DETAILED DESCRIPTION

[0056] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0057] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0058] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0059] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0060] Example 1

[0061] like Figure 1 As shown, this embodiment provides a test case generation method based on a large model and retrieval enhanced generation, including:

[0062] S1: Obtain software documentation and perform text segmentation and vectorization, and use all vectorized data to build a software knowledge base;

[0063] S2: Obtain the requirement document and vectorize it, and recall and reference the content in the software knowledge base based on the retrieval enhancement generation framework to obtain the optimized requirement document;

[0064] S3: construct a first Prompt instruction, input the optimized requirement document into at least one large language model, and use the first Prompt instruction as a prompt word to split the optimized requirement document into requirement fragments of several test points;

[0065] S4: Construct a second Prompt instruction for each test point requirement fragment, use each of the second Prompt instructions as a prompt word of at least one large language model, generate test cases corresponding to each test point, establish a mapping relationship between the test points and the test cases for linkage output.

[0066] In the specific implementation process, most of the current solutions directly generate test cases from requirements. For example, if a requirement is very large (including a lot of content), a large number of test cases are directly generated at one time, which is not friendly enough for testers. At the same time, some modules or test points can be modified, and updates cannot be made according to some modules. This embodiment provides a method that can refine a large model to test points. At the same time, the test points and test cases are always mapped and associated while being displayed on a dual-screen linkage, so that it is convenient for testers to view and further operate or modify, such as making partial updates or generating test cases multiple times, which is more portable.

[0067] Specifically, first obtain the software description document and perform text segmentation and vectorization, and use all the vectorized data to build a software knowledge base;

[0068] Then, the requirement document is obtained and vectorized, and the content in the referenced software knowledge base is recalled based on the retrieval enhancement generation framework to obtain the optimized requirement document;

[0069] Then, a first Prompt instruction is constructed, the optimized requirement document is input into at least one large language model, and the first Prompt instruction is used as a prompt word to split the optimized requirement document into requirement fragments of several test points;

[0070] Finally, a second prompt instruction is constructed for each test point requirement fragment, and each second prompt instruction is used as a prompt word of at least one large language model to generate test cases corresponding to each test point, and a mapping relationship is established between the test point and the test case for linkage output;

[0071] The generated test cases can be further formatted and then exported to the test management platform, where users can view, edit and improve the generated test cases;

[0072] This method generates test cases based on RAG and LLM, which has significant advantages. It not only improves the efficiency and quality of test case writing and reduces the professional skill requirements of testers, but also supports diverse test case formats and scenarios, improves test case management and collaboration efficiency, and has strong expansion capabilities. This method helps to promote technological innovation and development in the field of software testing, and has important practical value and broad application prospects.

[0073] Example 2

[0074] This embodiment provides a test case generation method based on a large model and retrieval enhanced generation, including:

[0075] S1: Obtain software documentation and perform text segmentation and vectorization, and use all vectorized data to build a software knowledge base;

[0076] The software description document includes: product introduction, background knowledge and functional description of each test point of the software;

[0077] S2: Obtain a requirement document and vectorize it, and recall and reference the content in the software knowledge base based on the retrieval enhancement generation framework to obtain an optimized requirement document; the requirement document includes: business rules for user operations and test points; the format of the requirement document includes any one of doc format, pdf format, txt format and markdown format;

[0078] S3: construct a first Prompt instruction, input the optimized requirement document into at least one large language model, and use the first Prompt instruction as a prompt word to split the optimized requirement document into requirement fragments of several test points;

[0079] S4: construct a second Prompt instruction for each requirement fragment of the test point, use each of the second Prompt instructions as a prompt word of at least one large language model, generate a test case corresponding to each test point, establish a mapping relationship between the test point and the test case for linkage output;

[0080] S5: Obtain user feedback information on the generated test case, and adjust the second Prompt instruction according to the feedback information; the feedback information includes any one or more of the following:

[0081] The user's check result on whether the optimal test case has ambiguity or vague description;

[0082] User feedback on the compatibility of the best test cases running on different platforms;

[0083] Users add or delete software test points;

[0084] User requirements for boundary test cases;

[0085] User demand for abnormal test cases;

[0086] Repeat step S4, regenerate a new test case according to the new second Prompt instruction, establish a new mapping relationship between the test point and the new test case for linkage output;

[0087] The formats of the linkage output include: any one or more of BDD format, manual test case format, Excel format and Xmind mind map format;

[0088] In step S3, the large language model includes: any one of GPT-4o, Claude-3.5, Gemini-1.5, GLM-4, QWen, ERNIE and Hunyuan.

[0089] In the specific implementation process, firstly, the software description document is obtained and the text fragments are segmented and vectorized, and then the software knowledge base is constructed using all the vectorized data;

[0090] like Figure 2 As shown in the figure, the user organizes the product introduction, background knowledge, specific function point introduction and other contents of the corresponding software platform into a software description document; then uploads the organized software description document to the platform; then slices the entire software description document data processing module into text segments one by one; then embeds each text segment separately through the Embedding model; finally, stores the vectorized data into a vector database to generate a software knowledge base;

[0091] This embodiment takes AiSee software as an example, and part of the text in its knowledge base is as follows:

[0092] "In today's era of information explosion, the voices of users are like a vast ocean of data, containing endless wisdom and value. These voices, whether positive praise or constructive criticism, are indispensable and valuable assets on the road to growth of every product. They are like mirrors, reflecting the strengths and weaknesses of our products, providing us with the direction and motivation for improvement. However, in the vast amount of user feedback, how can we effectively filter out valuable information? How can we deeply explore the deep connotation of user needs? How can we transform these feedbacks into actual product improvement strategies? These issues have become challenges faced by many businesses.

[0093] AiSee (love feedback) platform was born to solve these problems. It is an intelligent user feedback management platform integrated with artificial intelligence technology, dedicated to helping businesses collect, analyze and apply user feedback more efficiently. Through AiSee, we can discover problems more promptly, grasp user needs more accurately, and gain insight into market dynamics, thereby continuously optimizing product experience and improving user satisfaction and loyalty. "

[0094] like Figure 3 As shown, the requirement documents of several test points of the software are then obtained and embedded vectorized. In this embodiment, the requirement document supports multiple formats, such as doc format, pdf format, txt format and markdown format. For users, the requirement document can be a requirement text, a doc file of a requirement document, or a requirement list of a requirement management platform, which is essentially the text data of a requirement document. If there is table data in the requirement document, the table data is uniformly converted into the table text format of markdown.

[0095] The requirements document in this embodiment is as follows:

[0096] “Requirement background: AiSee platform needs to support several new dimension alarms. These new dimensions are based on user dimension, city dimension, and problem scenario dimension.

[0097] User dimension: User ID dimension, supports configuring alarms to monitor multiple feedbacks from the same user in a short period of time.

[0098] City dimension: supports monitoring the rapid growth of user feedback in individual cities.

[0099] Problem dimension: Supports alerting when the number of users reporting issues with the same semantics increases rapidly.

[0100] Based on the retrieval enhancement generation framework, the content in the reference software knowledge base is recalled, the vectorized data of the requirement document is queried in the vector database, and relevant text fragments are recalled; and the relevant specific software function descriptions are supplemented to the content in the requirement document, so as to generate an optimized requirement document;

[0101] Then, a first Prompt instruction is constructed, the optimized requirement document is input into at least one large language model, and the first Prompt instruction is used as a prompt word to split the optimized requirement document into requirement fragments of several test points;

[0102] This embodiment designs a special task to split the test point, that is, designs a special prompt (first prompt instruction) to split the test point and the module; this prompt specifies a chain of thought of manually written use cases to let the large model think about the splitting, chain of thought (Chain-of-Thought, CoT), which is a technology used to improve the reasoning ability of the large language model (LLM). The model first gradually derives a series of intermediate steps or sub-goals, which constitute a "chain of thought";

[0103] For example, a thought chain can be: in-depth word-by-word analysis of the requirement document ==> confirm the requirement details ==> confirm the boundaries ==> split the modules and test points ==> output in a specific format; the first prompt example is as follows:

[0104] "As a senior software testing expert, the steps for writing test cases include the following:

[0105] 1. Understand the needs;

[0106] Requirement document analysis: Read and understand the requirement document in detail to clarify the specific requirements of each functional module.

[0107] Discuss with stakeholders: Discuss the requirements with product managers, developers, and other stakeholders to ensure nothing is missed or misunderstood.

[0108] 2. Determine the test strategy;

[0109] Select test method: Select appropriate test method (such as functional testing, performance testing, automated testing, etc.) according to requirements and project characteristics.

[0110] Determine the test scope: clarify which functions and modules need to be tested and which do not. "

[0111] Then, according to the content of the requirement fragment of each test point and the test case type selected by the user, select the appropriate prompt template. For example, if the user chooses to generate a test case in BDD format, select the corresponding prompt template; import the improved requirement document content into the selected prompt template to generate the second prompt instruction corresponding to each test point;

[0112] like Figure 4 As shown, at least one large language model is then selected. If multiple large language models are selected, different weights need to be assigned to each large model. After the test case is generated, the optimal test case is selected according to the weight; the second Prompt instruction is used as the prompt word for each large language model to generate a structured test case;

[0113] Finally, the structured test cases are mapped to each test point. On the one hand, the test points are highlighted, and on the other hand, the test points are mapped to the test cases (visually linked with lines). The test cases are then displayed in BDD format. The mapped test cases are output to the test management platform for dual-screen linkage display (one screen displays the test points, and the other screen displays the corresponding test cases. When sliding up or down on either side of the screen, the test points and test cases are always associated together due to the existence of lines). Users can view, edit and improve the generated test cases on the platform.

[0114] like Figure 5 and 6 As shown in the figure, the test cases generated by referencing the knowledge base and not referencing the knowledge base are shown respectively. It can be seen that based on the retrieval enhancement generation technology, the test cases generated after referencing the knowledge base are more comprehensive and accurate. Therefore, this method can quickly generate test cases corresponding to the requirement document without the need for user historical data, and output test cases in a format close to the user's habits, generating better results.

[0115] like Figure 7 As shown, the method also includes multiple rounds of iterative generation of new test cases, and through multiple interactions and feedback, the test cases are continuously improved and optimized to ensure their comprehensiveness, accuracy and applicability; the method in this embodiment can not only generate initial test cases, but also check, correct and supplement them in subsequent multiple rounds of interactions; specific scenarios include:

[0116] 1) Initial test case generation: In the first round of interaction, the initial version of test cases is generated based on the requirements document and functional description provided by the user; these test cases cover basic functional tests and major usage scenarios; rapid generation of preliminary test cases lays the foundation for subsequent multiple rounds of optimization;

[0117] 2) Check and correct test case ambiguity: In subsequent interactions, check whether the generated test cases have ambiguity or vague descriptions; users can point out unclear points through the feedback mechanism, and the system will adjust and optimize based on the feedback; eliminate ambiguity in test cases to ensure that testers can accurately understand and execute test steps;

[0118] 3) Supplement compatibility test cases: Automatically generate compatibility test cases based on user feedback and needs to ensure the compatibility of the software on different operating systems, browsers and devices; cover more test scenarios to ensure the stability and consistency of the software in various environments;

[0119] 4) Test cases for new functional points: In multiple rounds of interaction, users can propose new functional points or requirements. This method generates corresponding test cases based on these new requirements; it can flexibly respond to changes in requirements and ensure that new functional points are fully tested;

[0120] 5) Optimize boundary test cases: Automatically generate and optimize boundary test cases to ensure the performance of the software under boundary conditions, so as to improve the test coverage and ensure the stability of the software under extreme conditions; for example, this method will generate test cases for boundary conditions such as the minimum value, maximum value, and null value of the input field;

[0121] 6) Improve exception handling test cases: Generate and improve test cases related to exception handling to ensure the software's ability to handle exceptions, so as to improve the robustness of the software and ensure that it can correctly handle and recover in exception situations;

[0122] Generate test cases based on multiple rounds of iterations. Each round can expand on the test cases generated in the previous round to continuously improve the test cases, ultimately achieving full coverage and high-precision test case generation;

[0123] This method generates test cases based on RAG and LLM, which has significant advantages. It not only improves the efficiency and quality of test case writing and reduces the professional skill requirements of testers, but also supports diverse test case formats and scenarios, improves test case management and collaboration efficiency, and has strong expansion capabilities. This method helps to promote technological innovation and development in the field of software testing, and has important practical value and broad application prospects.

[0124] Example 3

[0125] like Figure 8 As shown, this embodiment provides a test case generation system based on a large model and retrieval enhanced generation, and applies a test case generation method based on a large model and retrieval enhanced generation described in Example 2, including:

[0126] Knowledge base construction unit 301: used to obtain software description documents and perform text segmentation and vectorization, and use all vectorized data to build a software knowledge base;

[0127] The requirement document optimization unit 302 is used to obtain the requirement document and vectorize it, and recall and reference the content in the software knowledge base based on the retrieval enhancement generation framework to obtain the optimized requirement document;

[0128] The requirement document splitting unit 303 is used to construct a first Prompt instruction, input the optimized requirement document into at least one large language model, and use the first Prompt instruction as a prompt word to split the optimized requirement document into requirement segments of a plurality of test points;

[0129] The test case generating unit 304 is used to construct a second Prompt instruction for each requirement fragment of each test point, use each of the second Prompt instructions as a prompt word of at least one large language model, generate a test case corresponding to each test point, establish a mapping relationship between the test point and the test case for linkage output;

[0130] The test case updating unit 305 is used to obtain user feedback information on the generated test case, and adjust the second Prompt instruction according to the feedback information;

[0131] The test case generating unit 304 is repeatedly executed to regenerate a new test case according to the new second Prompt instruction, and a new mapping relationship is established between the test point and the new test case for linkage output.

[0132] In the specific implementation process, first, the knowledge base construction unit 301 obtains the software description document and performs text segmentation and vectorization, and uses all the vectorized data to construct the software knowledge base;

[0133] Next, the requirement document optimization unit 302 obtains the requirement document and vectorizes it, and recalls and references the content in the software knowledge base based on the retrieval enhancement generation framework to obtain an optimized requirement document;

[0134] Then the requirement document splitting unit 303 constructs a first Prompt instruction, inputs the optimized requirement document into at least one large language model, and uses the first Prompt instruction as a prompt word to split the optimized requirement document into requirement segments of a plurality of test points;

[0135] Then, the test case generating unit 304 is used to construct a second Prompt instruction for each requirement fragment of the test point, use each of the second Prompt instructions as a prompt word of at least one large language model, generate a test case corresponding to each test point, and establish a mapping relationship between the test point and the test case for linkage output;

[0136] Finally, the test case updating unit 305 obtains the user's feedback information on the generated test case, and adjusts the second Prompt instruction according to the feedback information; repeatedly executes the test case generating unit 304, regenerates a new test case according to the new second Prompt instruction, establishes a new mapping relationship between the test point and the new test case for linkage output, thereby generating a more complete test case;

[0137] This system generates test cases based on RAG and LLM, which has significant advantages. It not only improves the efficiency and quality of test case writing and reduces the professional skill requirements of testers, but also supports a variety of test case formats and scenarios, improves the efficiency of test case management and collaboration, and has strong expansion capabilities.

[0138] Example 4

[0139] like Fig. 9 As shown, this embodiment provides a test case generation platform based on a large model and retrieval enhancement generation, and applies a test case generation method based on a large model and retrieval enhancement generation described in Example 2, including:

[0140] The UI layer is located at the top level and is used to implement user interaction;

[0141] The application layer is located in the middle layer and is used to implement various functions;

[0142] The LLM layer, located at the bottom layer, is used to integrate several large language models.

[0143] In the specific implementation process, 1) the UI layer mainly provides user interaction functions, including BDD test case generation, manual test case generation, multi-round test case generation, and knowledge base;

[0144] Requirements and use case mapping: Requirements are decomposed into test points, and the mapping between test points and use cases is displayed;

[0145] BDD test case generation: supports the generation of test cases that conform to the behavior-driven development (BDD) format, helping users better understand and describe requirements;

[0146] Test case update: Supports updating test cases in BBD test case nodes. Users can expand and supplement existing test cases to ensure the comprehensiveness and accuracy of test cases.

[0147] Knowledge base management: Provides knowledge base addition, management and reference functions. Users can upload and manage the background knowledge and function point introduction of the software platform to improve the relevance and accuracy of test case generation;

[0148] The UI layer design fully considers the diverse needs of users and supports a variety of test case formats and output forms, reducing the user's manual conversion costs and improving the user experience;

[0149] 2) In the application layer:

[0150] Prompt Engineering: Through few-shot learning and various test case writing methods, we enrich the PromptEngineering module so that large models can fully understand requirements and output stable results;

[0151] Multi-round Prompt: Encapsulates scenarios for generating test cases in multiple rounds. Users can easily click to generate test cases in specific fields, improving generation efficiency.

[0152] Dynamic Prompt: Supports dynamic updating and matching of prompt content to ensure that the system can adapt to the latest test case requirements at any time;

[0153] Fine-Tuning: Fine-tuning some large models to ensure that they can stably understand and perform specific tasks and improve the accuracy of input and output;

[0154] Historical data memory: record and manage the user's historical requirement documents, so that users can find and supplement test cases in specific fields at any time;

[0155] Data processing chain: Improve the stability and parsing accuracy of generated test cases through the data processing chain, and ensure that the output conforms to the format specified by the user;

[0156] The application layer uses a variety of technical means to ensure that large models can generate test cases efficiently and accurately, and provide flexible expansion and update capabilities;

[0157] 3) In the LLM layer, the large model factory mode: designed as a large model factory mode, it supports the encapsulation and use of multiple large models, and facilitates the switching of trial and generation environments;

[0158] Model selection and calling: Automatically select the appropriate large model for calling according to task requirements to ensure the accuracy and stability of the generated results;

[0159] Encapsulation interface: Decoupling of large models from upper-layer application scenarios is achieved through encapsulation interfaces, ensuring flexibility and scalability of the system;

[0160] This platform has the following advantages:

[0161] 1) Improve the efficiency of test case writing:

[0162] By automatically generating structured test cases, the efficiency of testers in writing test cases is significantly improved, saving a lot of time. According to the actual use results, this platform can save 10% to 70% of the testers' time, which will help shorten the software development cycle and improve the overall R&D efficiency.

[0163] 2) Reduce the professional skills requirements of testers:

[0164] This platform uses big model technology to automatically generate test cases, reducing the professional knowledge and skills required of testers; testers only need to select appropriate inputs according to the requirements document, and the platform can automatically generate corresponding test cases; this will reduce the training cost and threshold for testers, allowing more people to participate in the writing of test cases;

[0165] 3) Improve the quality of test cases:

[0166] The test cases generated based on the big model technology have high accuracy and availability. By introducing a mechanism for generating test cases in multiple rounds, the generated test cases can better meet actual needs, improve the quality and coverage of test cases, and ensure the comprehensiveness and effectiveness of the test.

[0167] 4) Support diverse test case formats and scenarios:

[0168] This platform supports multiple test case formats, such as BDD (behavior-driven development) test case format, manual test case format, etc., as well as multiple output formats, such as Excel, Xmind mind map, etc. In addition, this platform also supports the generation of multiple types of test cases such as extended test case boundaries and abnormal scenarios to meet the testing requirements in different scenarios;

[0169] 5) Powerful expansion capabilities:

[0170] This platform adopts the large model factory mode and supports the packaging and use of multiple large models, which makes it have strong expansion capabilities and can adapt to the ever-changing technical environment and business needs; users can select or replace large models according to actual needs to ensure that the platform always maintains the best performance;

[0171] 6) Dynamic recall related business knowledge:

[0172] The RAG technology is used to dynamically recall the background knowledge in the knowledge base of related business products, so that the large language model can fully understand the functional knowledge points and proper nouns in the related fields. This method ensures that the generated test cases have high relevance and accuracy, further improving the quality of test cases.

[0173] 7) Multi-round use case generation dialogue:

[0174] A multi-round test case generation chain is designed, which can expand and supplement the existing generated cases. Through the idea of ​​context, the large language model has a clearer purpose when generating test cases, achieves higher accuracy and recall rate, and ensures the comprehensiveness and completeness of the test cases.

[0175] The same or similar reference numerals correspond to the same or similar components;

[0176] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0177] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A test case generation method based on large model and retrieval enhanced generation, characterized in that: include: S1: Obtain software documentation and perform text segmentation and vectorization, and use all vectorized data to build a software knowledge base; S2: Obtain the requirement document and quantize it, recall and reference the content in the software knowledge base based on the retrieval enhancement generation framework, and obtain the optimized requirement document; S3: construct a first Prompt instruction, input the optimized requirement document into at least one large language model, and use the first Prompt instruction as a prompt word to split the optimized requirement document into requirement fragments of several test points; S4: Construct a second Prompt instruction for each test point requirement fragment, use each of the second Prompt instructions as a prompt word of at least one large language model, generate test cases corresponding to each test point, establish a mapping relationship between the test points and the test cases for linkage output.

2. A test case generation method based on large model and retrieval enhanced generation according to claim 1, characterized in that: The method further comprises: S5: Obtaining user feedback information on the generated test case, and adjusting the second Prompt instruction according to the feedback information; Repeat step S4, regenerate a new test case according to the new second Prompt instruction, establish a new mapping relationship between the test point and the new test case for linkage output.

3. A test case generation method based on large model and retrieval enhanced generation according to claim 1 or 2, characterized in that: The formats of linkage output include: BDD format, manual test case format, Excel format and Xmind mind map format.

4. A test case generation method based on large model and retrieval enhanced generation according to claim 2, characterized in that: In step S5, the feedback information includes any one or more of the following: The user's check result on whether the optimal test case has ambiguity or vague description; User feedback on the compatibility of the best test cases running on different platforms; The user adds or deletes software test points; User requirements for boundary test cases; User demand for exceptional test cases.

5. The test case generation method based on large model and retrieval enhanced generation according to claim 1 is characterized in that: In step S1, the software description document includes: product introduction, background knowledge and functional description of each test point of the software.

6. A test case generation method based on large model and retrieval enhanced generation according to claim 1, characterized in that: In step S2, the format of the demand document includes any one of doc format, pdf format, txt format and markdown format.

7. A test case generation method based on large model and retrieval enhanced generation according to claim 1, characterized in that: In step S2, the requirement document includes: business rules for user operations and test points.

8. A test case generation method based on large model and retrieval enhanced generation according to claim 1, characterized in that: In step S3, the large language model includes any one or more of GPT-4o, Claude-3.5, Gemini-1.5, GLM-4, QWen, ERNIE and Hunyuan.

9. A test case generation system based on a large model and retrieval enhanced generation, applying a test case generation method based on a large model and retrieval enhanced generation as described in any one of claims 1 to 8, characterized in that: include: Knowledge base construction unit: used to obtain software description documents and perform text segmentation and vectorization, and use all vectorized data to build a software knowledge base; Requirement document optimization unit: used to obtain and vectorize the requirement document, recall and reference the content in the software knowledge base based on the retrieval enhancement generation framework, and obtain the optimized requirement document; A requirement document splitting unit is used to construct a first Prompt instruction, input the optimized requirement document into at least one large language model, and use the first Prompt instruction as a prompt word to split the optimized requirement document into requirement fragments of a plurality of test points; Test case generation unit: used to construct a second Prompt instruction for the requirement fragment of each test point, use each of the second Prompt instructions as the prompt word of at least one large language model, generate test cases corresponding to each test point, establish a mapping relationship between the test point and the test case for linkage output.

10. A test case generation system based on large model and retrieval enhanced generation according to claim 9, characterized in that: The system further comprises: A test case updating unit: used to obtain user feedback information on the generated test case, and adjust the second Prompt instruction according to the feedback information; The test case generation unit is executed repeatedly, a new test case is regenerated according to the new second Prompt instruction, and a new mapping relationship is established between the test point and the new test case for linkage output.

Images