A method and system for generating functional test cases based on large language model

Abstract

The present invention discloses a method and system for generating functional test cases based on a large language model, the method comprising: obtaining a requirement document, and parsing the requirement document into an intermediate representation of a tree structure; traversing the tree path of the intermediate representation to extract fine-grained requirement points, and obtaining all fine-grained requirement points; designing prompt words for functional test cases according to the fine-grained requirement points, and generating functional test cases according to the prompt words using a large language model; performing post-processing according to the functional test cases, extracting fields of the functional test cases, and generating final functional test case results. Through the technical solution of the present invention, changes in requirements can be synchronized quickly, and each fine-grained requirement point can be extracted to generate functional test cases in a targeted manner, reducing the complexity of writing requirement documents, and saving time and manpower.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a method for generating functional test cases based on a large language model and a system for generating functional test cases based on a large language model. Background Art

[0002] Automated functional test case generation is a method of creating functional test cases through automated tools and techniques, which can improve the efficiency and coverage of software testing. The existing technology includes the following testing methods, but each has its own shortcomings, as follows:

[0003] (1) Solutions based on traditional natural language processing technology: Requirements documents are prone to contain ambiguous, vague or imprecise descriptions, and traditional NLP (Natural Language Processing) technology is difficult to accurately capture these details or implicit contexts. When dealing with complex sentence structures and logical reasoning, traditional NLP technology is prone to misunderstanding, resulting in the generated test cases not meeting actual requirements.

[0004] (2) Solution based on behavior-driven development: Detailed requirements descriptions need to be written at the beginning of the project, and these descriptions must use standardized languages ​​(such as Gherkin). For projects that lack specification documents or change frequently, the cost of writing and maintaining these requirements documents is high. The description language of behavior-driven development is more suitable for simple business scenario descriptions. For complex business logic and interaction scenarios, using behavior-driven development to write test descriptions may result in lengthy requirements documents that are difficult to maintain.

[0005] (3) Model-based solution: Building and maintaining a model that reflects the requirements document is a time-consuming and technically intensive process. Especially when the system requirements are complex or change frequently, the cost of updating the model is high. When the system functions change, the model needs to be updated synchronously. Especially in projects with frequent requirements changes, maintaining multiple complex test models may bring great workload and technical burden. Therefore, the model-based solution is not optimal in terms of resource consumption, labor cost, and real-time requirements.

[0006] In summary, when the requirements document description is not standardized, the business logic and interaction scenarios are complex, and the requirements change frequently, the above solution cannot meet the requirements of fast generation, cost reduction and efficiency improvement, accurate generation, and resource saving. Summary of the invention

[0007] In view of the above problems, the present invention provides a method and system for generating functional test cases based on a large language model, which parses the requirement document and converts it into a tree-shaped intermediate representation, extracts all fine-grained requirement points from the intermediate representation, and generates functional test cases using a large language model by designing prompt words and splicing them to obtain the final functional test case results. The generated functional test cases can quickly synchronize the changes in requirements, and can extract each fine-grained requirement point to generate functional test cases in a targeted manner, thereby reducing the complexity of writing requirement documents and saving time and manpower.

[0008] To achieve the above object, the present invention provides a method for generating functional test cases based on a large language model, comprising:

[0009] Obtaining a requirement document, and parsing the requirement document into an intermediate representation of a tree structure;

[0010] Traversing the tree path of the intermediate representation to extract fine-grained demand points, and obtaining all fine-grained demand points;

[0011] Designing prompt words for functional test cases according to the fine-grained demand points, and generating functional test cases using a large language model based on the prompt words;

[0012] Post-processing is performed according to the functional test case, fields of the functional test case are extracted, and a final functional test case result is generated.

[0013] In the above technical solution, preferably, the process of parsing the requirement document into an intermediate representation of a tree structure specifically includes:

[0014] According to the association relationship between the title level of the requirement document and the ordinary text, the requirement document is parsed into an intermediate representation of a tree structure.

[0015] In the above technical solution, preferably, the traversing the tree path of the intermediate representation to extract fine-grained demand points to obtain all fine-grained demand points, the specific process includes:

[0016] Traversing the tree path of the intermediate representation from the root node to each leaf node;

[0017] Concatenate the titles and texts of all nodes on each tree path to obtain a fine-grained demand point;

[0018] All fine-grained demand points corresponding to all tree paths are stored in a list.

[0019] In the above technical solution, preferably, the prompt words of the functional test case are designed according to the fine-grained demand points, and the functional test case is generated by a large language model according to the prompt words. The specific process includes:

[0020] Design prompt words for extracting functional test cases from the fine-grained demand points, and fill each fine-grained demand point in the prompt word to fill each fine-grained demand point in the list as a prompt word of the large language model;

[0021] A functional test case is generated by the large language model according to the prompt word.

[0022] In the above technical solution, preferably, the post-processing is performed according to the functional test case, the fields of the functional test case are extracted, and the final functional test case result is generated, and the specific process includes:

[0023] Post-processing the functional test cases to extract fields contained in each functional test case, wherein the fields include module name, title, prerequisite, test steps, expected results and case attributes;

[0024] All extracted functional test case fields are concatenated to generate the final functional test case results.

[0025] The present invention also proposes a functional test case generation system based on a large language model, which uses the functional test case generation method based on a large language model disclosed in any one of the above technical solutions, including:

[0026] A requirement document parsing module, used to obtain a requirement document and parse the requirement document into an intermediate representation of a tree structure;

[0027] A demand traversal extraction module, used to traverse the tree path of the intermediate representation to extract fine-grained demand points and obtain all fine-grained demand points;

[0028] A test case generation module, used to design prompt words for functional test cases according to the fine-grained demand points, and to generate functional test cases using a large language model based on the prompt words;

[0029] The final result generating module is used to perform post-processing according to the functional test case, extract the fields of the functional test case, and generate the final functional test case result.

[0030] In the above technical solution, preferably, the requirement document parsing module is specifically used for:

[0031] According to the association relationship between the title level of the requirement document and the ordinary text, the requirement document is parsed into an intermediate representation of a tree structure.

[0032] In the above technical solution, preferably, the demand traversal extraction module is specifically used to:

[0033] Traversing the tree path of the intermediate representation from the root node to each leaf node;

[0034] Concatenate the titles and texts of all nodes on each tree path to obtain a fine-grained demand point;

[0035] All fine-grained demand points corresponding to all tree paths are stored in a list.

[0036] In the above technical solution, preferably, the test case generation module is specifically used to:

[0037] Fill each fine-grained demand point in the list as a prompt word for the large language model;

[0038] A functional test case is generated by the large language model according to the prompt word.

[0039] In the above technical solution, preferably, the final result generating module is specifically used for:

[0040] Post-processing the functional test cases to extract fields contained in each functional test case, wherein the fields include module name, title, prerequisite, test steps, expected results and case attributes;

[0041] All extracted functional test case fields are concatenated to generate the final functional test case results.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows: by parsing the requirement document and converting it into a tree-shaped intermediate representation, all fine-grained requirement points are extracted from the intermediate representation, and functional test cases are generated and spliced ​​using a large language model by designing prompt words to obtain the final functional test case results. The generated functional test cases can quickly synchronize the changes in requirements, and can extract each fine-grained requirement point to generate functional test cases in a targeted manner, thereby reducing the complexity of writing requirement documents and saving time and manpower. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a flow chart of a method for generating functional test cases based on a large language model disclosed in an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of an intermediate representation of a tree structure disclosed in an embodiment of the present invention;

[0045] Figure 3 A module diagram of a functional test case generation system based on a large language model disclosed in an embodiment of the present invention.

[0046] In the figure, the corresponding relationship between each component and the reference numeral is as follows:

[0047] 1. Requirement document parsing module, 2. Requirement traversal extraction module, 3. Test case generation module, 4. Final result generation module. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0050] like Figure 1 and Figure 2 As shown, a method for generating functional test cases based on a large language model according to the present invention includes:

[0051] Obtain the requirement document and parse it into an intermediate representation of a tree structure;

[0052] Traverse the tree path of the intermediate representation to extract fine-grained demand points and obtain all fine-grained demand points;

[0053] Design prompt words for functional test cases based on fine-grained demand points, and generate functional test cases based on the prompt words using a large language model;

[0054] Post-process the functional test cases, extract the fields of the functional test cases, and generate the final functional test case results.

[0055] In this implementation, the requirement document is parsed and converted into a tree-like intermediate representation, all fine-grained requirement points are extracted from the intermediate representation, and functional test cases are generated and spliced ​​using a large language model by designing prompt words to obtain the final functional test case results. The generated functional test cases can quickly synchronize changes in requirements and can extract each fine-grained requirement point to generate functional test cases in a targeted manner, thereby reducing the complexity of writing requirement documents and saving time and manpower.

[0056] Specifically, during the requirement document writing phase, the first-level title corresponding to the requirement point is required to be "Software System Functional Requirements". Specific functional points are written in this chapter, starting from the second-level title to list specific functional points. You can add third-level, fourth-level, etc. sub-titles as needed. There is no limit on the number of functional points, the level of the title, or the length of the document. All content in this chapter will be used to extract the requirement points and generate functional test cases.

[0057] Among them, Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text.

[0058] In the above implementation, preferably, the requirement document is parsed into an intermediate representation of a tree structure, and the specific process includes:

[0059] According to the association between the title level of the requirement document and the ordinary text, the requirement document is parsed into an intermediate representation of a tree structure.

[0060] Specifically, the requirements document is parsed into a forest structure, with each first-level title corresponding to a tree. The first-level title corresponds to the root node of a tree, and each title without a subtitle corresponds to a leaf node. Each title (as long as it is not ordinary text) corresponds to a leaf node on the tree. The leaf node is a dictionary that contains the title content, the level of the title, and the ordinary text between the title and the next title. In this way, the forest tree can be used to represent the relationship between titles and texts at all levels in the entire requirements document. After obtaining the forest structure after parsing the entire requirements document, only focus on the tree with the first-level title "Software System Functional Requirements". This tree is the intermediate representation for obtaining all fine-grained requirements points next. This reduces the complexity of writing requirements documents and saves time and manpower.

[0061] In the above implementation, preferably, the tree path of the intermediate representation is traversed to extract fine-grained demand points to obtain all fine-grained demand points. The specific process includes:

[0062] Traverse the intermediate representation of the tree path from the root node to each leaf node;

[0063] Concatenate the headings and texts of all nodes on each tree path to obtain a fine-grained demand point;

[0064] All fine-grained demand points corresponding to all tree paths are stored in a list.

[0065] In the above implementation, preferably, prompt words of functional test cases are designed according to fine-grained demand points, and a large language model is used to generate functional test cases according to the prompt words. The specific process includes:

[0066] Design prompt words for extracting functional test cases from fine-grained requirement points, and fill each fine-grained requirement point in the prompt word. Fill each fine-grained requirement point in the list as a prompt word for the large language model;

[0067] Generate functional test cases from a large language model based on prompt words.

[0068] Specifically, when requirements change, only the changed sections can be selected for regeneration. Only the corresponding fine-grained requirement point content needs to be replaced and input into the large language model, which enables the generated functional test cases to quickly synchronize the changes in requirements.

[0069] In the above implementation, preferably, post-processing is performed according to the functional test case, the fields of the functional test case are extracted, and the final functional test case result is generated. The specific process includes:

[0070] Post-process the functional test cases to extract the fields contained in each functional test case, where the fields include module name, title, prerequisite, test steps, expected results and case attributes;

[0071] All extracted functional test case fields are concatenated to generate the final functional test case results.

[0072] like Figure 3 As shown, the present invention also proposes a functional test case generation system based on a large language model, applying the functional test case generation method based on a large language model disclosed in any one of the above embodiments, including:

[0073] The requirement document parsing module 1 is used to obtain the requirement document and parse the requirement document into an intermediate representation of a tree structure;

[0074] Demand traversal extraction module 2, used to traverse the tree path of the intermediate representation to extract fine-grained demand points and obtain all fine-grained demand points;

[0075] Test case generation module 3, used to design prompt words for functional test cases according to fine-grained demand points, and generate functional test cases based on the prompt words using a large language model;

[0076] The final result generating module 4 is used to perform post-processing according to the functional test case, extract the fields of the functional test case, and generate the final functional test case result.

[0077] In the above implementation, preferably, the requirement document parsing module 1 is specifically used for:

[0078] According to the association between the title level of the requirement document and the ordinary text, the requirement document is parsed into an intermediate representation of a tree structure.

[0079] In the above implementation, preferably, the demand traversal extraction module 2 is specifically used for:

[0080] Traverse the intermediate representation of the tree path from the root node to each leaf node;

[0081] Concatenate the titles and texts of all nodes on each tree path to obtain a fine-grained demand point;

[0082] All fine-grained demand points corresponding to all tree paths are stored in a list.

[0083] In the above implementation, preferably, the test case generation module 3 is specifically used for:

[0084] Fill in each fine-grained demand point in the list as a prompt word for the large language model;

[0085] Generate functional test cases from a large language model based on prompt words.

[0086] In the above implementation, preferably, the final result generating module 4 is specifically used for:

[0087] Post-process the functional test cases to extract the fields contained in each functional test case, where the fields include module name, title, prerequisite, test steps, expected results and case attributes;

[0088] All extracted functional test case fields are concatenated to generate the final functional test case results.

[0089] According to the functional test case generation system based on a large language model disclosed in the above-mentioned implementation mode, the functions to be implemented by each module thereof correspond to the respective steps of the functional test case generation method based on a large language model disclosed in the above-mentioned implementation mode. During the implementation process, operations are performed with reference to the above-mentioned implementation mode, which will not be repeated here.

[0090] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating functional test cases based on a large language model, characterized in that: include: Obtaining a requirement document, and parsing the requirement document into an intermediate representation of a tree structure; Traversing the tree path of the intermediate representation to extract fine-grained demand points, and obtaining all fine-grained demand points; Designing prompt words for functional test cases according to the fine-grained demand points, and generating functional test cases using a large language model based on the prompt words; Post-processing is performed according to the functional test case, fields of the functional test case are extracted, and a final functional test case result is generated.

2. The method for generating functional test cases based on a large language model according to claim 1, characterized in that: The specific process of parsing the requirement document into an intermediate representation of a tree structure includes: According to the association relationship between the title level of the requirement document and the ordinary text, the requirement document is parsed into an intermediate representation of a tree structure.

3. The method for generating functional test cases based on a large language model according to claim 2, characterized in that: The tree path of the intermediate representation is traversed to extract fine-grained demand points to obtain all fine-grained demand points. The specific process includes: Traversing the tree path of the intermediate representation from the root node to each leaf node; Concatenate the titles and texts of all nodes on each tree path to obtain a fine-grained demand point; All fine-grained demand points corresponding to all tree paths are stored in a list.

4. The method for generating functional test cases based on a large language model according to claim 3, characterized in that: The process of designing prompt words for functional test cases according to the fine-grained demand points, and generating functional test cases using a large language model according to the prompt words, specifically includes: Design prompt words for extracting functional test cases from the fine-grained demand points, and fill each fine-grained demand point in the prompt word to fill each fine-grained demand point in the list as a prompt word of the large language model; A functional test case is generated by the large language model according to the prompt word.

5. The method for generating functional test cases based on a large language model according to claim 4, characterized in that: The post-processing according to the functional test case, extracting the fields of the functional test case, and generating the final functional test case result, specifically includes: Post-processing the functional test cases to extract fields contained in each functional test case, wherein the fields include module name, title, prerequisite, test steps, expected results and case attributes; All extracted functional test case fields are concatenated to generate the final functional test case results.

6. A functional test case generation system based on a large language model, characterized in that: The method for generating functional test cases based on a large language model according to any one of claims 1 to 5 is applied, comprising: A requirement document parsing module, used to obtain a requirement document and parse the requirement document into an intermediate representation of a tree structure; A demand traversal extraction module, used to traverse the tree path of the intermediate representation to extract fine-grained demand points and obtain all fine-grained demand points; A test case generation module, used to design prompt words for functional test cases according to the fine-grained demand points, and to generate functional test cases using a large language model based on the prompt words; The final result generating module is used to perform post-processing according to the functional test case, extract the fields of the functional test case, and generate the final functional test case result.

7. The functional test case generation system based on a large language model according to claim 6, characterized in that: The requirement document parsing module is specifically used for: According to the association relationship between the title level of the requirement document and the ordinary text, the requirement document is parsed into an intermediate representation of a tree structure.

8. The functional test case generation system based on a large language model according to claim 7, characterized in that: The demand traversal extraction module is specifically used for: Traversing the tree path of the intermediate representation from the root node to each leaf node; Concatenate the titles and texts of all nodes on each tree path to obtain a fine-grained demand point; All fine-grained demand points corresponding to all tree paths are stored in a list.

9. The functional test case generation system based on a large language model according to claim 8, characterized in that: The test case generation module is specifically used for: Fill each fine-grained demand point in the list as a prompt word for the large language model; A functional test case is generated by the large language model according to the prompt word.

10. The functional test case generation system based on a large language model according to claim 9, characterized in that: The final result generation module is specifically used for: Post-processing the functional test cases to extract fields contained in each functional test case, wherein the fields include module name, title, prerequisite, test steps, expected results and case attributes; All extracted functional test case fields are concatenated to generate the final functional test case results.

Images