Method and device for generating test case based on large model in software test

Abstract

The invention relates to the technical field of automatic generation of test cases, and particularly provides a method and a device for generating a test case based on a large model in software testing, firstly, a webpage DOM (Document Object Model) tree acquirer uses a large language model tool compiled by python and selenium technologies to acquire a webpage DOM tree; the method comprises the following steps: automatically accessing and acquiring DOM tree information of a url page provided by a user, transmitting the crawled DOM tree information to a large language model, then retrieving a professional domain knowledge base by using an RAG retrieval enhancement generation technology according to questions of the user, and transmitting the retrieved related domain knowledge to the large language model, and the big language model automatically generates a corresponding test case according to the requirement of a natural language of a user through the obtained DOM tree information and related knowledge retrieved from the professional domain knowledge base. Compared with the prior art, the working efficiency can be improved to the maximum extent, the accuracy of the test case is improved, and finally the purpose of improving the product quality is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic test case generation, and specifically provides a method and device for generating test cases based on a large model in software testing. Background Art

[0002] With the increasing complexity of software systems and shortened development cycles, traditional manual test case construction methods are struggling to meet the testing needs of modern software systems. Automated testing, as an effective solution, can significantly improve testing efficiency and ensure software quality. However, existing test cases, especially complex ones, are almost entirely manually constructed. Manually written test cases present the following challenges that need to be addressed:

[0003] (1) Excessive subjectivity;

[0004] Different testers may write test cases based on their own understanding and experience, which may lead to differences between test cases and even omission of certain key test points. Subjective judgment may also affect the priority and importance of test cases, causing some important test scenarios to be overlooked.

[0005] (2) Efficiency issues;

[0006] Manually writing test cases takes a lot of time and effort, especially when dealing with complex and large systems. This process can become very time-consuming. As the system is constantly iterated and updated, the test cases also need to be modified and expanded accordingly, which further increases the workload.

[0007] (3) Poor consistency;

[0008] Due to the lack of unified standards and specifications, test cases written by different testers may have differences in format, structure, and content, which affects the maintenance and sharing of test cases. Inconsistent test cases may also lead to confusion and misunderstandings during test execution.

[0009] (4) Risk of omission;

[0010] Even experienced testers may miss certain test points or test scenarios due to negligence or knowledge gaps. The risk of such omissions is particularly high when dealing with complex and dynamic systems.

[0011] (5) Difficulty in updating and maintaining;

[0012] As software systems are constantly iterating and updated, test cases also need to be modified and expanded accordingly. However, due to the large number of test cases and their disparate locations in different documents, this process can become tedious and time-consuming. Furthermore, if test cases are not updated and maintained promptly, this can lead to a decrease in test coverage and reduced test quality.

[0013] (6) Lack of automation support;

[0014] Manually written test cases are often not directly usable for automated testing, which limits further improvements in test efficiency and accuracy. While test cases can be converted into automated scripts, this process requires additional investment and time. Summary of the Invention

[0015] The present invention aims to address the deficiencies of the above-mentioned prior art and provides a method for generating test cases based on a large model in software testing with strong practicality.

[0016] A further technical task of the present invention is to provide a device for generating test cases based on a large model in software testing that is rationally designed, safe and applicable.

[0017] The technical solution adopted by the present invention to solve its technical problem is:

[0018] A method for generating test cases based on a big model in software testing. First, a web page DOM tree acquirer uses a big language model tool written in Python and Selenium technology to automatically access and obtain the DOM tree information of the URL page provided by the user and pass the crawled DOM tree information to the big language model. Then, based on the user's question, the professional domain knowledge base is retrieved using RAG retrieval enhancement generation technology, and the retrieved relevant domain knowledge is passed to the big language model. The big language model automatically generates corresponding test cases according to the user's natural language requirements based on the acquired DOM tree information and the relevant knowledge retrieved from the professional domain knowledge base.

[0019] Furthermore, users need to prepare a server with a GPU, first install Python, then deploy the relevant large language model OpenAI_API service, install the embedding model and vector knowledge base milvus for local deployment, or use it online by accessing the corresponding URL.

[0020] Furthermore, when a user sends a request to the OpenAi_API service of the large language model, the system will follow the following steps to help the user generate a use case:

[0021] (1) The large language model first calls the tool - the web page DOM tree obtainer;

[0022] (2) Use RAG technology to create professional knowledge base and retrieve relevant knowledge;

[0023] (3) Large model generation use case.

[0024] Furthermore, in step (1), a UI automation script written in Python and selenium technology is used to access the URL web page provided by the user, and then the crawler technology is used to obtain the DOM tree of the page, and the obtained DOM tree information is regularly trimmed, and finally the streamlined web page DOM tree information is passed to the large model.

[0025] Furthermore, in step (2), the user first needs to prepare relevant knowledge documents. After the user uploads the document, the user document information is parsed and the document content is segmented according to the delimiter set by the user. After the document is segmented, the corresponding document slices are vectorized using a vector model, and finally the vectorized document slices are stored in the vector database.

[0026] When a user asks a question to the large language model, the vector database is first searched, and the best matching document slices are calculated based on the vectors and returned to the large model.

[0027] Furthermore, the knowledge base creation and retrieval process is implemented using python+haystack+milvus, where python and haystack are used to implement document parsing and slicing, the embedding vector model uses Qwen3-embedding-8B, and the vector database uses milvus.

[0028] Furthermore, in step (3), based on the DOM tree information obtained by the web page DOM tree obtainer and the expert knowledge base information obtained by the retrieval enhancement generation technology of the above-mentioned knowledge base, according to the user's questions, combined with the prompt word engineering, using natural language, answer the relevant questions raised by the user, and automatically generate test cases that meet the specific requirements of the user's questions.

[0029] Furthermore, the specific test cases required are as follows:

[0030] 1) Through the first-tier component: the webpage DOM tree acquirer, access the webpage provided by the user, obtain the DOM tree information of the webpage and return it to the large language model;

[0031] 2) Through the second-tier component: the professional domain knowledge base, based on the user's question, the professional domain vector knowledge base is retrieved using RAG enhanced retrieval generation technology, and the retrieved relevant professional domain knowledge is passed to the large language model for subsequent generation;

[0032] 3) Using the large language model, the DOM tree information obtained in 1) and the relevant knowledge in the professional knowledge base documents retrieved in 2) are combined with the relevant prompt word engineering to answer user-related questions in natural language and generate test cases that meet user requirements, and then return them to the user interaction page.

[0033] A device for generating test cases based on a large model in software testing, comprising: at least one memory and at least one processor;

[0034] The at least one memory is configured to store a machine-readable program;

[0035] The at least one processor is configured to call the machine-readable program to execute a method for generating test cases based on a large model in software testing.

[0036] Compared with the prior art, the method and device for generating test cases based on a large model in software testing of the present invention have the following outstanding beneficial effects:

[0037] The present invention can realize the automatic generation of test cases, and can generate test cases that meet the requirements according to the needs proposed by the user and the knowledge base of relevant professional fields, and can effectively solve the problems of artificially generated test cases being too subjective, inefficient, poorly consistent, having the risk of omissions, and difficult to update and maintain. It can greatly improve work efficiency, enhance the accuracy of test cases, ensure the coverage of test cases, improve the accuracy of testers performing tests to a great extent, ensure the quality of the company's products, and greatly save manpower and material resources in the whole link during the test case writing process. Compared with the method of manually writing test cases, the workload is reduced and the benefits are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 It is a flowchart of a method for generating test cases based on a large model in software testing;

[0040] Figure 2 This is an interface diagram (1) in an embodiment of the present invention;

[0041] Figure 3 This is the interface diagram (2) in the embodiment of the present invention;

[0042] Figure 4 This is the interface diagram (3) in the embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention will be further described in detail below in conjunction with specific embodiments. Obviously, the embodiments described are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0044] A best embodiment is given below:

[0045] like Figure 1 As shown, in this embodiment, a method for generating test cases based on a large model in software testing is used. The web page DOM tree acquirer uses a large language model tool written in Python and Selenium technology. The tool can automatically access and obtain the DOM tree information of the URL page provided by the user and pass the crawled DOM tree information to the large language model.

[0046] Then, based on the user's question, RAG (retrieval-augmented generation) technology is used to search the professional domain knowledge base, and the retrieved relevant domain knowledge is passed to the large language model. The large language model automatically generates corresponding test cases based on the user's natural language requirements through the obtained DOM tree information and the relevant knowledge retrieved from the professional domain knowledge base.

[0047] Users need to prepare a server with a GPU, first install Python, then deploy the relevant large language model OpenAI_API service, install the embedding model and vector knowledge base milvus for local deployment, or use it online by accessing the corresponding URL.

[0048] When a user sends a request to the OpenAi_API service for a large language model, the system follows the following steps to help the user generate a use case:

[0049] (1) The large language model first calls the tool - the web page DOM tree obtainer;

[0050] The web page DOM tree acquirer is mainly written in Python and selenium technology to access the URL web page provided by the user in the form of UI automation script, and then uses crawler technology to obtain the DOM tree of the page, and regularly cuts the obtained DOM tree information, and finally passes the streamlined web page DOM tree information to the large model.

[0051] (2) Use RAG technology to create professional knowledge base and retrieve relevant knowledge;

[0052] Users first need to prepare relevant knowledge documents. After uploading the document, the system parses the document information and segments the document content according to the user's specified delimiters. After segmentation, the system uses a vector model to vectorize the corresponding document slices and finally stores the vectorized document slices in a vector database. When a user asks a question to the large language model, the vector database is first searched, and the best matching document slices are calculated based on the vectors and returned to the large model.

[0053] The above knowledge base creation and retrieval process is implemented using python+haystack+milvus, where python and haystack are used to implement document parsing and slicing, Qwen3-embedding-8B is used as the embedding vector model, and milvus is used as the vector database.

[0054] (3) Large model generation use case;

[0055] Based on the DOM tree information obtained by the web page DOM tree obtainer and the expert knowledge base information obtained by the retrieval enhancement generation technology of the above-mentioned knowledge base, according to the user's questions, combined with the prompt word engineering, using natural language to answer relevant questions raised by the user, automatically generate test cases that meet the specific requirements of the user's questions.

[0056] The specific required test cases are:

[0057] 1) Through the first-tier component: the web page DOM tree acquirer, access the web page provided by the user, obtain the DOM tree information of the web page, and return it to the large language model.

[0058] 2) Through the second-layer component: professional domain knowledge base, based on the user's question, RAG (augmented retrieval generation) technology is used to retrieve the professional domain vector knowledge base, and the retrieved relevant professional domain knowledge is passed to the large language model for subsequent generation.

[0059] 3) Using the large language model, the DOM tree information obtained in 1) and the relevant knowledge in the professional knowledge base documents retrieved in 2) are combined with the relevant prompt word engineering to answer user-related questions in natural language and generate test cases that meet user requirements, and then return them to the user interaction page.

[0060] The relevant prompt words are as follows:

[0061] You are a senior test architect in the ERP field. [Role Introduction] and [Abilities] are as follows. Please design and generate test cases based on the requirements provided by users. The requirements for case generation are as follows:

[0062] Character Introduction

[0063] -language:Chinese / English

[0064] -description: Expert in advanced test design for ERP systems

[0065] -Background: 10 years of experience in ERP testing, ISTQB certified expert, software evaluator, certified public accountant

[0066] -Personality: rigorous and meticulous, strong logic, and risk-sensitive

[0067] -Expertise: test strategy formulation, scenario modeling, and exception flow coverage.

[0068] Possessing abilities;

[0069] 1. Test design capabilities

[0070] -Equivalence class division: accurately identify valid equivalence classes and invalid equivalence classes;

[0071] -Boundary value analysis: select the boundary value of input or output for testing;

[0072] -Scenario analysis: Build a user journey map to identify key paths;

[0073] -Orthogonal decomposition: handles multi-parameter combination scenarios;

[0074] -State Migration: Verify complex state transition logic.

[0075] 2. Technical support capabilities

[0076] -Automation framework: Design an extensible testing framework;

[0077] -Quality measurement: define coverage indicator system;

[0078] - Defect prediction: Apply machine learning to predict risk modules;

[0079] -Performance modeling: building load capacity models;

[0080] Use case design requirements

[0081] 1. Design principles

[0082] -MECE principle: The set of use cases is mutually independent and completely exhaustive;

[0083] - Risk priority: Allocate test intensity according to the impact of failure;

[0084] - Traceability: Establish a requirements-use case bidirectional traceability matrix;

[0085] -Maintainability: adopt modular design mode;

[0086] 2. Implementation Guidelines:

[0087] - Minimum prerequisites: no more than 3 key items

[0088] -Atomic operation: a single use case verifies a single functional point.

[0089] 3. Format constraints:

[0090] - Clear results: Each expected result contains a verifiable assertion

[0091] -Priority definition: P0 (blocking) / P1 (high) / P2 (medium) / P3 (low)

[0092] process:

[0093] -Objective: You need to design a zero-missing test case based on the requirements design document uploaded in the attachment, covering quality characteristics such as functionality, performance, security, usability, reliability, compatibility, and portability;

[0094] Step 1: Analyze the requirements document and establish test points;

[0095] Step 2: Generate test cases for this requirement based on the test points;

[0096] Step 3: Supplement: equivalence class, boundary value, reverse process, abnormal process, performance, security, ease of use, reliability, compatibility, and portability use cases.

[0097] Expected results:

[0098] 1. Achieve path coverage > 95%, with as many use cases as possible, without omissions, and output in full;

[0099] 2. The proportion of positive functional use cases shall not exceed 60%, and the proportion of equivalence classes, boundary values, reverse processes, abnormal processes, performance, security, usability, reliability, compatibility, and portability use cases shall not be less than 40%.

[0100] Output format:

[0101] 1. Please imitate the JSON format to output plain text, but do not start with json and do not generate json code segments

[0102] 2. The output requirements for each field are as follows:

[0103] [{

[0104] "Number": "For use cases generated in the same batch, use the same 4-digit random uppercase English letter followed by a 2-digit number, which increases starting from 01."

[0105] "Brief description": "(1) First output the function menu name, such as [XXXX], and do not mistake the quality characteristics for the function menu name; (2) Then, concisely describe the use case scenario."

[0106] "Quality Characteristics": Please select from the following 7 items: Function / Performance / Security / Usability / Compatibility / Reliability / Portability. If there is no match or you are unsure, please fill in 'Function'.

[0107] "Process Type": Please select from the three options: Forward Process / Reverse Process / Exception Process. If there is no match, leave it blank. Do not enter any value other than these three options.

[0108] "Preconditions": "The conditions that need to be prepared in advance to execute this use case. If there are no conditions, you can leave it blank."

[0109] "Details": {

[0110] "Verification Point": "First output the function menu name, such as [XXXX], and then output the specific verification point"

[0111] "Expected results": "The expected effect to be achieved".

[0112] 3. Example description:

[0113] "serial number":"……"

[0114] "Brief description": "..."

[0115] "Process type" "..."

[0116] Prerequisites:

[0117] "Quality Characteristics": "..."

[0118] Details:

[0119] "Verification point": "..."

[0120] "Expected result": "..."

[0121] "Verification point": "..."

[0122] "Expected result": "..."

[0123] …]

[0124] …

[0125] Based on the above method, a device for generating test cases based on a large model in software testing in this embodiment includes: at least one memory and at least one processor;

[0126] The at least one memory is configured to store a machine-readable program;

[0127] The at least one processor is configured to call the machine-readable program to execute a method for generating test cases based on a large model in software testing.

[0128] The above-mentioned specific implementation methods are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementation methods. Any technical solutions that conform to the above-mentioned specific implementation methods of the present invention and any appropriate changes or substitutions made thereto by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0129] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating test cases based on a large model in software testing, characterized in that: First, the web page DOM tree acquirer uses a large language model tool written in Python and Selenium technology to automatically access and obtain the DOM tree information of the URL page provided by the user and pass the crawled DOM tree information to the large language model. Then, based on the user's question, the professional domain knowledge base is retrieved using RAG retrieval enhancement generation technology, and the retrieved relevant domain knowledge is passed to the large language model. The large language model automatically generates corresponding test cases based on the user's natural language requirements through the obtained DOM tree information and the relevant knowledge retrieved from the professional domain knowledge base.

2. The method for generating test cases based on a large model in software testing according to claim 1, characterized in that: Users need to prepare a server with a GPU, first install Python, then deploy the relevant large language model OpenAI_API service, install the embedding model and vector knowledge base milvus for local deployment, or use it online by accessing the corresponding URL.

3. The method for generating test cases based on a large model in software testing according to claim 2, characterized in that: When a user sends a request to the OpenAi_API service for a large language model, the system follows the following steps to help the user generate a use case: (1) The large language model first calls the tool - the web page DOM tree obtainer; (2) Use RAG technology to create professional knowledge base and retrieve relevant knowledge; (3) Large model generation use case.

4. The method for generating test cases based on a large model in software testing according to claim 3, characterized in that: In step (1), a UI automation script written in Python and selenium technology is used to access the URL web page provided by the user, and then the crawler technology is used to obtain the DOM tree of the page, and the obtained DOM tree information is trimmed in a regular manner, and finally the streamlined web page DOM tree information is passed to the large model.

5. The method for generating test cases based on a large model in software testing according to claim 4, characterized in that: In step (2), the user first needs to prepare relevant knowledge documents. After the user uploads the document, the user document information is parsed and the document content is segmented according to the delimiter set by the user. After the document is segmented, the corresponding document slices are vectorized using the vector model, and finally the vectorized document slices are stored in the vector database. When a user asks a question to the large language model, the vector database is first searched, and the best matching document slices are calculated based on the vectors and returned to the large model.

6. The method for generating test cases based on a large model in software testing according to claim 5, characterized in that: The knowledge base creation and retrieval process is implemented using python+haystack+milvus, where python and haystack are used to implement document parsing and slicing, the embedding vector model uses Qwen3-embedding-8B, and the vector database uses milvus.

7. The method for generating test cases based on a large model in software testing according to claim 6, characterized in that: In step (3), based on the DOM tree information obtained by the web page DOM tree obtainer and the expert knowledge base information obtained by the retrieval enhancement generation technology of the above-mentioned knowledge base, according to the user's questions, combined with the prompt word engineering, using natural language to answer the relevant questions raised by the user, automatically generate test cases that meet the specific requirements of the user's questions.

8. The method for generating test cases based on a large model in software testing according to claim 7, characterized in that: The specific required test cases are as follows: 1) Through the first-tier component: the webpage DOM tree acquirer, access the webpage provided by the user, obtain the DOM tree information of the webpage and return it to the large language model; 2) Through the second-tier component: the professional domain knowledge base, based on the user's question, the professional domain vector knowledge base is retrieved using RAG enhanced retrieval generation technology, and the retrieved relevant professional domain knowledge is passed to the large language model for subsequent generation; 3) Using the large language model, the DOM tree information obtained in 1) and the relevant knowledge in the professional knowledge base documents retrieved in 2) are combined with the relevant prompt word engineering to answer user-related questions in natural language and generate test cases that meet user requirements, and then return them to the user interaction page.

9. A device for generating test cases based on a large model in software testing, characterized in that: include: at least one memory and at least one processor; The at least one memory is configured to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 8.

Images