Test case generation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a test case generation method and device, equipment and a storage medium. The method specifically comprises the steps that a natural language test requirement prompt input by a user is received; based on a natural language test demand prompt, retrieving function description associated information from the plug-in knowledge base through a retrieval enhancement generation (RAG) technology; wherein the plug-in knowledge base comprises a function description knowledge base and a UI element knowledge base; function description documents for describing function business logic are stored in the function description knowledge base; the UI element knowledge base stores positioning modes and element attributes of interface elements; obtaining a structured natural language test step according to the pre-trained large language model, the natural language test demand prompt and the function description associated information; and based on the natural language test step and the natural language test requirement, prompting the positioning mode of the interface element retrieved from the UI element knowledge base, and generating a test case.

Description

Technical Field

[0001] This invention relates to the field of testing technology, and in particular to a test case generation method, apparatus, device, and storage medium. Background Technology

[0002] Currently, when generating test cases, the script generation method based on interface operation recording is usually adopted. That is, testers simulate business processes by manually operating the software interface, and the automated tools record the operation trajectory and convert it into script code.

[0003] However, this approach has the following problems: First, it relies heavily on manual operation, resulting in low script generation efficiency and high labor costs. Each change in requirements requires re-recording, which limits the test coverage to a few preset paths. Second, it lacks intelligent understanding capabilities and cannot parse test intent from functional descriptions or natural language requirements, making it difficult to achieve truly intelligent test case generation. Summary of the Invention

[0004] The main objective of this invention is to provide a test case generation method, apparatus, device, and storage medium, which aims to solve the problems of low script generation efficiency, high labor costs, and small test coverage caused by the high dependence on manual operation in the existing test case generation process, as well as the lack of intelligent understanding ability, which makes it impossible to parse test intent from functional descriptions or natural language requirements, and thus makes it difficult to achieve intelligent test case generation.

[0005] In a first aspect, embodiments of this disclosure provide a test case generation method, including: Receive natural language test request prompts from the user; Based on the natural language testing requirements, functional description related information is retrieved from the plug-in knowledge base using the Retrieval Enhancement Generation (RAG) technology. The plug-in knowledge base includes a functional description knowledge base and a UI element knowledge base. The functional description knowledge base stores functional description documents that describe the business logic of the functions. These functional description documents include at least one of a product requirement document and a product prototype annotation document. The UI element knowledge base stores the positioning methods and element attributes of interface elements. Based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information, obtain structured natural language testing steps; Based on the natural language testing steps and the location method of the interface elements retrieved from the UI element knowledge base based on the natural language testing requirements, test cases are generated.

[0006] Secondly, embodiments of this disclosure provide a test case generation apparatus, comprising: The receiving unit is used to receive natural language test requirement prompts input by the user; The retrieval unit is used to retrieve function description-related information from the plug-in knowledge base based on the natural language testing requirement prompts and through retrieval enhancement generation RAG technology; wherein, the plug-in knowledge base includes a function description knowledge base and a UI element knowledge base; the function description knowledge base stores function description documents describing the business logic of the function; the function description documents include at least one of a product requirement document and a product prototype annotation description document; the UI element knowledge base stores the positioning methods and element attributes of interface elements; The acquisition unit is used to acquire structured natural language testing steps based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information. The generation unit is used to generate test cases based on the natural language testing steps and the location method of the interface elements retrieved from the UI element knowledge base based on the natural language testing requirements.

[0007] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the method described in the first aspect above.

[0008] Fourthly, embodiments of this disclosure provide a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this disclosure provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0010] The at least one technical solution provided by the embodiments of the present invention can achieve the following technical effects: In this embodiment of the invention, by receiving user natural language test requirement prompts and intelligently retrieving relevant information about the function description from an external knowledge base containing a function description knowledge base and a UI element knowledge base based on RAG technology, a pre-trained large language model is used to generate structured natural language test steps. Finally, executable test cases are automatically generated by combining UI element location methods. This embodiment of the invention can transform the traditional static and passive recorded test case generation into dynamic and intelligent semantic-driven test case generation. This effectively avoids the difficulties in script maintenance caused by low efficiency of manual operation and frequent requirement changes, while significantly improving the efficiency and scenario coverage of test case generation, fundamentally ensuring the automation and intelligence level of software testing. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is one of the flowcharts illustrating a test case generation method provided in an embodiment of the present invention; Figure 2 This is a second flowchart illustrating a test case generation method provided in one embodiment of the present invention. Figure 3 A schematic diagram of the module composition of a test case generation device 300 provided in one embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0014] Please see Figure 1 , Figure 1 This is a flowchart illustrating a test case generation method provided in one embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps: Step 102: Receive the user's input of natural language test requirements.

[0015] Step 104: Based on natural language testing requirement prompts, retrieve relevant information about function descriptions from the plug-in knowledge base using retrieval enhancement generation (RAG) technology; wherein, the plug-in knowledge base includes a function description knowledge base and a UI element knowledge base; the function description knowledge base stores function description documents that describe the business logic of the function; the function description documents include at least one of the product requirement document and the product prototype annotation description document; the UI element knowledge base stores the positioning methods and element attributes of interface elements.

[0016] Step 106: Based on the pre-trained large language model, natural language testing requirements, and functional description information, obtain the structured natural language testing steps.

[0017] Step 108: Generate test cases based on the natural language testing steps and the location methods of interface elements retrieved from the UI element knowledge base based on the natural language testing requirement prompts.

[0018] In one embodiment of the invention, user input of natural language test request prompts can first be received through multiple channels. These channels may include web-based graphical interfaces, RESTful API interfaces, command-line tools, and integration interfaces with other development tools. This embodiment can also support multiple input formats, such as plain text, voice input (after speech recognition conversion), and structured text descriptions.

[0019] When users submit test requests, multi-layered validation and processing can be performed. First, the input text undergoes basic grammar checking to ensure there are no obvious grammatical errors or spelling issues. Next, key terms and test-related vocabulary in the text can be identified and mapped to testing-related terminology. For example, the user-entered word "login" will be mapped to the standard term "user login".

[0020] The preprocessing stage can also include text standardization and normalization. It can identify and handle synonyms and near-synonyms to ensure consistency in subsequent processing. For example, expressions such as "user registration," "new user account opening," and "account registration" will be standardized to "user registration process." This process can be implemented based on a domain-specific thesaurus and semantic understanding model.

[0021] To improve processing accuracy, contextual information can be added to the input text, including metadata such as user identity information, test environment configuration, timestamps, and project background. This information helps to better understand the specific scenarios and constraints of the test requirements.

[0022] This embodiment also supports multi-turn conversational input of requirements, allowing users to gradually refine their test requirements through continuous natural language dialogue. This embodiment can also maintain the dialogue state, remembering previous interactions and ensuring consistency in subsequent processing. This is particularly suitable for complex testing scenarios, allowing users to gradually clarify test requirements through multiple interactions.

[0023] In one embodiment of the present invention, after input processing is completed, the core information retrieval stage can be entered. This stage employs RAG (Retrieval-augmented Generation) technology to retrieve functional description information related to the test requirements from an external knowledge base, providing rich semantic background for the generation of subsequent test steps.

[0024] The external knowledge base can specifically include a functional specification knowledge base and a UI element knowledge base. The functional specification knowledge base can store functional specification documents describing the business logic of the functions; these documents can include at least one of a product requirements document and a product prototype annotation specification document. The UI element knowledge base can store the positioning methods and element attributes of interface elements.

[0025] Specifically, the external knowledge base can be a distributed knowledge storage system composed of multiple specialized sub-repositories. The functional specification knowledge base can store various functional specification documents describing the business logic of functions, which may include at least one of the following: product requirement documents, product prototype annotation documents (such as design specifications), user manuals, and API documents. These documents may exist in multiple formats such as HTML, PDF, Word, and Markdown, and can be uniformly parsed and indexed.

[0026] The UI element knowledge base specifically stores information about user interface elements, including element positioning methods (such as CSS selectors and XPath expressions), element attributes (such as type, name, and state), and relationships between elements. The UI element knowledge base also records the historical usage and change history of elements, providing a reference for element selection.

[0027] In one example, in addition to the aforementioned functional description knowledge base and UI element knowledge base, the plug-in knowledge base can also include a historical test case knowledge base and an auxiliary knowledge base. The historical test case knowledge base can store previously generated test cases and their execution results, including detailed information such as test pass rates, discovered defects, and execution times. This historical data can provide valuable experience for generating new tests. The auxiliary knowledge base can integrate industry standard documents, best practice guidelines, compliance requirements, and other supplementary information to further enrich the knowledge base.

[0028] In one embodiment of the present invention, after obtaining the natural language test requirement prompts input by the user, the user input can be standardized by using a preset prompt word template, thereby improving the accuracy of RAG retrieval.

[0029] This embodiment can standardize natural language test requirement prompts based on preset prompt word templates to obtain standardized test requirement prompts. The prompt word templates are configured to guide users to describe the function to be tested in a structured way, as well as the specified operations of the function to be tested, in natural language text.

[0030] Specifically, firstly, the natural language test requirement prompts input by users can be standardized based on preset prompt word templates, thereby eliminating ambiguity and incompleteness in user descriptions and ensuring the stability of subsequent retrieval and generation processes. To ensure the stable performance of RAG technology in practical deployment, standardized design can be used to set prompt words.

[0031] In one example, the design principles of the prompt words may include at least one of the following: Clear objective: clearly state the functional module / scenario to be tested; Verb-driven: use terms such as "test...process" or "verify...function" to clearly state the behavioral intent; Optional constraints: browser environment, language, platform and other conditions can be added; Natural semantics: close to spoken language and maintain the integrity of the context.

[0032] Based on the design principles of the above prompts, the standard prompt template can be set as follows: Please generate automated test cases for the function name, which includes key functional items. Please list the complete test path, including page navigation, input data, operation steps, and expected results.

[0033] This standardization process generates standardized test requirement prompts corresponding to user input. User input such as "test user registration process" is transformed into standardized structured prompts, such as "Please generate automated test cases for my user registration process, which includes key functionalities such as mobile phone number verification and password setting. Please list the complete test path, including page redirection, input data, operation steps, and expected results." This provides a clear semantic foundation for subsequent retrieval.

[0034] After standardizing user input to obtain standardized test requirement prompts, the relevant information on function descriptions can be retrieved from the plug-in knowledge base based on the standardized test requirement prompts through the retrieval enhancement generation RAG technology.

[0035] In one embodiment of the present invention, when retrieving function description-related information from an external knowledge base using RAG technology, a query vector can be generated first. A pre-trained text embedding model, such as BERT or GPT (based on Transformer), can be used to convert the user-input natural language test requirement prompts into a high-dimensional vector representation, thus obtaining the query vector. The query vector can capture the deep semantic features of the text, rather than just surface-level word matching.

[0036] During vectorization, the global semantics and local emphasis of the text can be considered to ensure that the generated query vector contains both the overall meaning and highlights key information. For longer input texts, a hierarchical processing strategy can be adopted, first understanding the overall structure and then focusing on the details.

[0037] After obtaining the query vector, the similarity between the query vector and the pre-stored text paragraph vectors in the functional specification knowledge base can be calculated. This allows for the retrieval of a specified number of target text paragraphs with the highest similarity from the functional specification knowledge base, which are then used as functional specification association information. The pre-stored text paragraph vectors are obtained by vectorizing the documents in the functional specification knowledge base using a text embedding model.

[0038] In this embodiment, similarity searches can be performed using the vector index of the functional description knowledge base. The documents in the functional description knowledge base have all been pre-processed into blocks and vectorized, with each text block corresponding to a vector representation. An optimized approximate nearest neighbor search algorithm can be used to improve retrieval efficiency while ensuring accuracy.

[0039] The retrieval process is not a simple single match, but rather a multi-level filtering and sorting process. This embodiment considers multiple similarity metrics simultaneously, including semantic similarity, lexical overlap, and contextual relevance, to comprehensively evaluate the relevance of each text block. Finally, a specified number of target text paragraphs with the highest similarity are selected as the retrieval results; the specified number is typically 3, 4, or 5.

[0040] In one embodiment, the retrieved target text paragraph can be concatenated with the user-input natural language test requirement prompt to obtain enhanced test requirement prompts. The concatenation process maintains the accuracy and completeness of the test requirements. For potentially ambiguous expressions, the retrieved document content can be consulted for clarification and confirmation. For example, if the user input mentions "quick login," but the retrieved requirement document clearly states "quick login via mobile phone verification code," a more precise expression can be used during concatenation.

[0041] The enhanced test requirement prompts not only include the original user intent but also incorporate relevant business context, technical constraints, and best practice information. This enhancement ensures that subsequent steps are based on sufficient information, improving the quality and accuracy of the generated results.

[0042] In one embodiment of the present invention, after obtaining enhanced test requirement prompts, a structured natural language testing procedure can be obtained by combining a pre-trained large language model with the retrieved functional description association information. This step transforms relatively vague user requirements into a specific, executable sequence of operations, laying the foundation for subsequent script generation.

[0043] In this embodiment, the large language model used has been specially trained and optimized to possess domain-specific knowledge in testing. The training data includes a large number of test cases, test scripts, requirements documents, and related technical literature, ensuring that the model can understand specific concepts and expressions in the testing domain.

[0044] In one example, training a large language model involves several steps. First, training data is prepared, including a vast amount of test cases, automated test scripts, product requirement documents, and technical literature related to software testing. This data covers a wide range of scenarios, from functional testing to exception handling, ensuring the model is exposed to rich testing concepts and business logic. Next, a multi-stage training strategy combining pre-training and domain-specific fine-tuning is employed. Initial pre-training builds basic language understanding capabilities based on a general corpus. Then, domain-adaptive fine-tuning uses test-specific data to iteratively optimize the model, focusing on key tasks such as test step generation, assertion design, and test case structure. Masked language modeling and sequence-to-sequence training techniques are used to enhance the model's semantic capture of test terms such as "boundary value analysis" and "equivalence class partitioning," as well as operational processes. Reinforcement learning mechanisms can also be introduced during training to optimize strategies based on feedback from generated test cases and real test results, further improving the model's accuracy and practicality. Finally, cross-validation and manual evaluation ensure the model possesses robust test scenario parsing capabilities, supporting the efficient transformation from fuzzy requirements to structured test steps.

[0045] The model can employ a multi-task learning framework, simultaneously learning multiple related tasks such as test step generation, test data generation, and assertion design. This learning approach enables the model to better understand the overall requirements of the testing task and generate more consistent test steps.

[0046] During inference, the model employs a constraint generation strategy to ensure that the output test steps conform to predefined structure and format requirements. For example, each test step must include a clear operation description and expected result, and there must be a logical order between the steps.

[0047] In one embodiment, the generation process begins with the identification of high-level test scenarios. The model first analyzes the enhanced test requirement cues to identify the main test objectives and scenario scope. For example, for the test requirements of the "user registration process," the model will identify that different scenarios need to be covered, such as normal registration, exception handling, and boundary cases.

[0048] Next, the model generates a detailed sequence of test steps for each test scenario. Each step contains three key elements: an operation description, test data, and expected results. The operation description explains the specific action to be performed on the interface, such as clicking a button, entering text, or selecting an option. The test data provides the specific values ​​required for the operation, such as a valid mobile phone number or a valid password. The expected results clearly state the status or behavior should occur after the operation.

[0049] During the generation process, the model pays special attention to the completeness and executability of the test steps. The step descriptions must be sufficiently detailed and precise to ensure accurate conversion into executable code. Simultaneously, there must be clear logical relationships between the steps to form a complete test flow.

[0050] In one example, the user's input natural language testing requirement prompt could be: "Test user registration process". Through retrieval enhancement using RAG technology and the mandatory standardization of the prompt word template, the final natural language testing steps output by the large model can be as follows: 1. Open the homepage, click the "Register" button, and you will be redirected to the registration page; 2. Enter a valid mobile phone number in the "Mobile Number" input box (e.g., 138****1234); 3. Clicking the "Get Verification Code" button will display a message saying "Verification code has been sent"; 4. Enter the 6-digit verification code you received; 5. Set a login password (the password must meet the complexity requirements); 6. Clicking the "Register" button will redirect you to the "Welcome Page" and display a "Registration Successful" message; 7. Verify whether automatic login is enabled and redirect to the user's homepage.

[0051] The above natural language results will be used as a testing step and subsequently used to generate automated test scripts.

[0052] In one example, the generated test steps can undergo multi-level verification and optimization. First, the logical consistency of the steps can be checked to ensure there are no contradictions or omissions. Next, a historical test case knowledge base can be consulted to compare the test steps for similar functions and verify their reasonableness.

[0053] For complex testing scenarios, an iterative optimization strategy may be adopted. First, a basic version of the test steps is generated, and then it is gradually improved and refined based on verification feedback. This iterative process can be repeated multiple times until a satisfactory result is achieved.

[0054] This embodiment also supports a hybrid generation mode with human intervention. Test engineers can review the generated test steps and provide feedback. Based on this feedback, this embodiment can adjust the generation strategy and gradually improve the generation quality.

[0055] In one embodiment of the present invention, after obtaining the structured natural language test steps, test cases can be generated based on the natural language test steps and the location methods of interface elements retrieved from the UI element knowledge base based on the natural language test requirement prompts. The obtained structured test steps can be transformed into standardized, directly executable automated test scripts. This process relies on deep collaboration between the large language model and the test case execution tool Playwright, and is combined with the reference and dynamic update mechanism of the UI element knowledge base to achieve accurate matching and grammatical compliance of the script content. The core technical aspects of this step include two aspects: first, translating the natural language test steps into automated test scripts in Pytest format; second, identifying and obtaining the page UI elements required for script execution, including input boxes, buttons, prompt boxes, etc., to ensure the integrity and stability of the automated operation path.

[0056] First, the large model, acting as the main logic controller for script generation, receives natural language test steps from the first stage. This model has been pre-trained and fine-tuned with a large corpus of Playwright-related code, enabling it to map structured language descriptions to Python scripts. Test steps can be fed into the large model sequentially, and the model, considering the context, translates them step-by-step into standard Pytest function structures, including fixtures, step functions, and assertions.

[0057] During script generation, the large model needs to locate and manipulate elements in the user interface, such as identifying the unique identifiers (selectors) of controls like the "phone number input box" and the "register button." If the UI element knowledge base already contains the selector information for these controls on the target page (e.g., CSS selectors, XPath paths, or ARIA tags), the large model can directly call the relevant element identifiers through an indexing mechanism, thus accurately inserting them into the generated script statements. For example, in the test step "Enter 13812345678 in the phone number input box," the model can reference existing selectors in the database, such as #input-phone, and automatically generate the following statement: await page.fill("#input-phone", "13812345678").

[0058] If the target UI element is not found in the UI element knowledge base (i.e., the location method of the UI element is determined based on the natural language testing requirements), then the page analysis mode can be executed to extract dynamic elements. This process leverages Playwright's page analysis capabilities to automatically identify and extract the location methods of UI elements.

[0059] The page analysis process based on Playwright's page analysis capabilities is as follows: First, based on the natural language testing requirements and structured natural language testing steps, the URL (uniform resource locator) of the target page corresponding to the function to be tested is determined. Then, the Playwright tool is controlled to open the target page according to the determined URL. During the opening process, a real user access environment can be simulated, including setting appropriate viewport size, user agent, network conditions, etc.

[0060] Once the page has loaded, the Playwright tool can be used to comprehensively scan the target page's DOM structure. Using natural language processing (NLP) technology, the text descriptions of the target interface elements from the NLP testing step are semantically matched with the text content and accessibility attributes of the elements in the DOM structure to identify and extract the location methods of the target interface elements. During the scan, not only standard HTML elements are considered, but also dynamically generated content based on JavaScript and various custom controls are identified. A multimodal matching strategy is employed when identifying target interface elements, simultaneously analyzing visual features, semantic attributes, structural relationships, and other information to ensure accurate identification. For elements with text descriptions, NLP technology is used for semantic matching, not just text matching.

[0061] The location methods and associated attributes of successfully extracted target UI elements can be incrementally updated to the UI element knowledge base as new entries. The identified target UI elements can be standardized before storage. Stored information can include metadata such as locator, element attributes, recognition time, and confidence score. This information provides a reference for subsequent use and supports continuous optimization based on user feedback. Then, based on the location methods of UI elements retrieved from the UI element knowledge base and the location methods of the extracted target UI elements, combined with natural language testing procedures, executable test cases can be generated.

[0062] In one embodiment of the present invention, after generating basic test cases, enhanced functionality may also be provided to extend and optimize the test cases based on ISTQB (International Software Testing Qualification Board), significantly improving the coverage and effectiveness of the tests.

[0063] In one embodiment, natural language testing steps can be analyzed based on the ISTQB software testing standard to obtain enhanced test steps that include abnormal paths and boundary values. During the analysis of natural language testing steps, equivalence class partitioning analysis and boundary value analysis can be performed. Equivalence class partitioning analysis can identify valid and invalid categories of input data, generating comprehensive test data for each input field; while boundary value analysis focuses on the boundary conditions of the input range, ensuring sufficient testing of boundary cases.

[0064] In this embodiment, after obtaining the above-mentioned basic test cases, the basic test cases can be enhanced by combining the following three types of key knowledge inputs: 1. UI Element Knowledge Base: This knowledge base not only contains the selector for each control on the page, but also records its control type (such as text box, dropdown list, radio button, checkbox, button, etc.) and semantic tags (such as "phone number", "verification code", "password"). Based on control type and semantic information, the large model can automatically identify which input fields require boundary value and equivalence class testing, and which operation processes have tolerable interruptions and repetitions in user behavior paths.

[0065] 2. Historical Use Case Knowledge Base: This knowledge base records common abnormal paths, combined input patterns, and result assertion methods in manual testing or other automated generation processes for similar functions or page components. Before generating enhancement scripts, the large model calls this knowledge base, using the RAG mechanism to retrieve similar contexts as auxiliary references for the current enhancement path design. For example, when handling the "CAPTCHA verification" scenario, script snippets from past use cases involving "CAPTCHA error," "CAPTCHA timeout," and "empty CAPTCHA" can be retrieved to guide the design of the currently generated enhancement path.

[0066] 3. ISTQB Embedding of Test Design Theory Knowledge: Test design principles from the ISTQB standard can be introduced during the pre-training phase, and the ability to identify and apply the following key theoretical content is strengthened during post-training: Equivalence class partitioning: For each input field, the large model can automatically generate equivalent valid / invalid value pairs. For example, a mobile phone number input can include: valid mobile phone number, invalid characters, and empty input. Boundary value analysis: For fields with length or numerical limits (such as password length, amount field), automatically construct minimum value, maximum value, out-of-bounds value, etc. input; Abnormal path simulation: such as interrupted submission, repeated clicking of the submit button, retrying after network interruption, skipping the CAPTCHA step, etc. State switching and redirection logic: such as "whether to automatically clear fields after returning to the homepage and then entering the registration page after registration", "whether to redirect the page after the operation times out", etc.

[0067] In one example, decision table test analysis can be performed on natural language testing steps to handle complex combinations of business rules and generate test scenarios that cover various combinations of conditions.

[0068] This embodiment can also perform error guessing analysis on natural language testing steps. Based on historical testing experience and common error patterns, it proactively infers potential defects and generates corresponding test cases. Furthermore, it can perform combinatorial test analysis, efficiently covering various combinations of multiple parameters through methods such as orthogonal experimental design.

[0069] In one embodiment, a risk-based testing approach can be introduced, prioritizing coverage of high-risk functional areas and scenarios. Risk assessment is based on multiple factors, including the importance of the function, the complexity of the changes, and historical defect data.

[0070] For high-risk features, more detailed and rigorous test cases can be generated, increasing the depth and breadth of testing. At the same time, appropriate test resources can be allocated to different risk levels to ensure optimal testing efficiency.

[0071] Risk analysis can also take into account the balance between business impact and technical complexity. It can refer to business priorities and technical implementation difficulty to formulate reasonable testing strategies and ensure that the maximum testing effect is achieved with limited resources.

[0072] In one embodiment of the invention, testing experience accumulated in a historical test case knowledge base can be fully utilized. When generating new test cases, historical test cases with similar functionality can be retrieved, referencing their testing approaches, data selection, and assertion design. Historical anomaly path test case fragments similar to the functionality described in the natural language test steps can be retrieved from the historical test case knowledge base using RAG technology. These retrieved fragments are then combined with the natural language test steps and the ISTQB software testing standard to serve as input context, guiding the generation of enhanced test steps that include anomaly paths and boundary values.

[0073] The retrieval process is based on semantic similarity calculations, not just keyword matching. It can analyze the deep semantics of the test scenario to find truly relevant historical use cases. The reference materials include various aspects such as testing strategies, data selection methods, and anomaly handling.

[0074] Historical use cases also provide a benchmark for evaluating new use cases. They allow for comparison of newly generated use cases with historical ones in terms of coverage, level of detail, and other metrics, ensuring that the quality of new use cases is not lower than historical levels. Furthermore, they identify shortcomings in historical use cases and allow for improvements in the new ones.

[0075] In this embodiment of the invention, by receiving user natural language test requirement prompts and intelligently retrieving relevant information about the function description from an external knowledge base containing a function description knowledge base and a UI element knowledge base based on RAG technology, a pre-trained large language model is used to generate structured natural language test steps. Finally, executable test cases are automatically generated by combining UI element location methods. This embodiment of the invention can transform the traditional static and passive recorded test case generation into dynamic and intelligent semantic-driven test case generation. This effectively avoids the difficulties in script maintenance caused by low efficiency of manual operation and frequent requirement changes, while significantly improving the efficiency and scenario coverage of test case generation, fundamentally ensuring the automation and intelligence level of software testing.

[0076] Figure 3 The test case generation device 300 shown can achieve Figure 1 The method described in the embodiment achieves the same technical effect, and can be specifically referred to in the above description. Figure 1 The description of test case generation in the illustrated embodiment will not be repeated here. The test case generation device 300 includes: The receiving unit 301 is used to receive natural language test requirement prompts input by the user; The retrieval unit 302 is used to retrieve function description-related information from the plug-in knowledge base based on the natural language testing requirement prompts and through retrieval enhancement generation RAG technology; wherein, the plug-in knowledge base includes a function description knowledge base and a UI element knowledge base; the function description knowledge base stores function description documents describing the business logic of the function; the function description documents include at least one of a product requirement document and a product prototype annotation description document; the UI element knowledge base stores the positioning method and element attributes of interface elements; The acquisition unit 303 is used to acquire structured natural language testing steps based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information. The generation unit 304 is used to generate test cases based on the natural language testing steps and the location method of the interface elements retrieved from the UI element knowledge base based on the natural language testing requirement prompts.

[0077] Optionally, the retrieval unit 302 is used for: Based on a preset prompt word template, the natural language test requirement prompts are standardized to obtain standardized test requirement prompts; the prompt word template is configured to guide users to describe the function to be tested in a structured way, as well as the natural language text of the specified operation items of the function to be tested. Based on the standardized test requirements, the function description related information is retrieved from the plug-in knowledge base using the enhanced RAG (Retrieval Enhancement and Generation) technology. The acquisition unit 303 is used for: Based on the pre-trained large language model, the standardized test requirement prompts, and the functional description association information, a structured natural language test procedure is generated.

[0078] Optionally, the retrieval unit 302 is used for: Using a pre-trained text embedding model, the natural language test requirement prompts are transformed into query vectors; By calculating the similarity between the query vector and the pre-stored text paragraph vectors in the functional description knowledge base, a specified number of target text paragraphs with the highest similarity are retrieved from the functional description knowledge base, and the target text paragraphs are used as the functional description association information; wherein, the pre-stored text paragraph vectors are obtained in advance by vectorizing the documents in the functional description knowledge base through the text embedding model.

[0079] Optionally, the acquisition unit 303 is used for: The target text paragraph is concatenated with the natural language testing requirement prompt to obtain the enhanced testing requirement prompt; Based on the enhanced testing requirements and the pre-trained large language model, obtain the structured natural language testing steps.

[0080] Optionally, the generation unit 304 is used for: If the location method of the target interface element is not found in the UI element knowledge base, the Playwright tool is scheduled to execute the page analysis process; the target interface element is the interface element determined based on the natural language testing requirement prompt. The execution page analysis process includes: Based on the natural language testing requirements and the structured natural language testing steps, determine the Uniform Resource Locator (URL) of the target page corresponding to the function to be tested; The Playwright tool is controlled to open the target page based on the determined Uniform Resource Locator; The Playwright tool is used to scan the Document Object Model (DOM) structure of the target page, and natural language processing technology is used to semantically match the text description of the target interface element in the natural language testing step with the text content and accessibility attributes of the elements in the DOM structure in order to identify and extract the positioning method of the target interface element. The location methods and associated attributes of the successfully extracted target interface elements are incrementally updated to the UI element knowledge base as new entries. Based on the location methods of interface elements retrieved from the UI element knowledge base and the location methods of the extracted target interface elements, and in conjunction with the natural language testing steps, executable test cases are generated.

[0081] Optionally, the device further includes ( Figure 3 (not shown in the image) Analysis unit 305 is used to analyze the natural language test steps based on the ISTQB software testing standard after the test cases are generated, and obtain enhanced test steps including abnormal paths and boundary values. The enhancement unit 306 is used to enhance the generated test cases according to the enhancement test steps to obtain enhanced test cases.

[0082] Optionally, the analysis unit 305 is used for: From the historical test case knowledge base, the RAG technology is used to retrieve historical abnormal path test case fragments that are similar to the functions described in the natural language test steps; The retrieved historical abnormal path test case fragments are combined with the natural language testing steps and the ISTQB software testing standard to serve as input context, guiding the generation of the enhanced testing steps that include abnormal paths and boundary values.

[0083] In this embodiment of the invention, by receiving user natural language test requirement prompts and intelligently retrieving relevant information about the function description from an external knowledge base containing a function description knowledge base and a UI element knowledge base based on RAG technology, a pre-trained large language model is used to generate structured natural language test steps. Finally, executable test cases are automatically generated by combining UI element location methods. This embodiment of the invention can transform the traditional static and passive recorded test case generation into dynamic and intelligent semantic-driven test case generation. This effectively avoids the difficulties in script maintenance caused by low efficiency of manual operation and frequent requirement changes, while significantly improving the efficiency and scenario coverage of test case generation, fundamentally ensuring the automation and intelligence level of software testing.

[0084] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0085] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0086] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0087] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a non-contiguous transfer configuration at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Receive natural language test request prompts from the user; Based on the natural language testing requirements, functional description related information is retrieved from the plug-in knowledge base using the Retrieval Enhancement Generation (RAG) technology. The plug-in knowledge base includes a functional description knowledge base and a UI element knowledge base. The functional description knowledge base stores functional description documents that describe the business logic of the functions. These functional description documents include at least one of a product requirement document and a product prototype annotation document. The UI element knowledge base stores the positioning methods and element attributes of interface elements. Based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information, obtain structured natural language testing steps; Based on the natural language testing steps and the location method of the interface elements retrieved from the UI element knowledge base based on the natural language testing requirements, test cases are generated.

[0088] The above is as stated in this application. Figure 1 The test case generation disclosed in the embodiments described above can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in one or more embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in one or more embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0089] The electronic device can also perform Figure 1 The method for generating test cases will not be described in detail here.

[0090] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The methods of the embodiments shown are not described in detail here.

[0091] This application also proposes a computer program product, which is stored in a storage medium and executed by at least one processor to implement... Figure 1 The methods of the embodiments shown are not described in detail here.

[0092] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0093] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this application should be included within the scope of protection of one or more embodiments of this application.

[0094] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0095] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in the embodiments of this application, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0097] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

Claims

1. A test case generation method, characterized in that, include: Receive natural language test request prompts from the user; Based on the natural language testing requirements, functional description related information is retrieved from the plug-in knowledge base using the Retrieval Enhancement Generation (RAG) technology. The plug-in knowledge base includes a functional description knowledge base and a UI element knowledge base. The functional description knowledge base stores functional description documents that describe the business logic of the functions. These functional description documents include at least one of a product requirement document and a product prototype annotation document. The UI element knowledge base stores the positioning methods and element attributes of interface elements. Based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information, obtain structured natural language testing steps; Based on the natural language testing steps and the location method of the interface elements retrieved from the UI element knowledge base based on the natural language testing requirements, test cases are generated.

2. The method according to claim 1, characterized in that, The process of retrieving function description-related information from the plug-in knowledge base based on the natural language test requirement prompts, using retrieval enhancement generation (RAG) technology, includes: Based on a preset prompt word template, the natural language test requirement prompts are standardized to obtain standardized test requirement prompts; the prompt word template is configured to guide users to describe the function to be tested in a structured way, as well as the natural language text of the specified operation items of the function to be tested. Based on the standardized test requirements, the function description related information is retrieved from the plug-in knowledge base using the enhanced RAG (Retrieval Enhancement and Generation) technology. The step of obtaining a structured natural language testing process based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information includes: Based on the pre-trained large language model, the standardized test requirement prompts, and the functional description association information, a structured natural language test procedure is generated.

3. The method according to claim 1, characterized in that, The process of retrieving function description-related information from the plug-in knowledge base based on the natural language test requirement prompts, using retrieval enhancement generation (RAG) technology, includes: Using a pre-trained text embedding model, the natural language test requirement prompts are transformed into query vectors; By calculating the similarity between the query vector and the pre-stored text paragraph vectors in the functional description knowledge base, a specified number of target text paragraphs with the highest similarity are retrieved from the functional description knowledge base, and the target text paragraphs are used as the functional description association information; wherein, the pre-stored text paragraph vectors are obtained in advance by vectorizing the documents in the functional description knowledge base through the text embedding model.

4. The method according to claim 3, characterized in that, The step of obtaining a structured natural language testing process based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information includes: The target text paragraph is concatenated with the natural language testing requirement prompt to obtain the enhanced testing requirement prompt; Based on the enhanced testing requirements and the pre-trained large language model, obtain the structured natural language testing steps.

5. The method according to claim 1, characterized in that, The step of generating test cases based on the natural language testing steps, and based on the location method of the interface elements retrieved from the UI element knowledge base according to the natural language testing requirements, includes: If the location method of the target interface element is not found in the UI element knowledge base, the Playwright tool is scheduled to execute the page analysis process; the target interface element is the interface element determined based on the natural language testing requirement prompt. The execution page analysis process includes: Based on the natural language testing requirements and the structured natural language testing steps, determine the Uniform Resource Locator (URL) of the target page corresponding to the function to be tested; The Playwright tool is controlled to open the target page based on the determined Uniform Resource Locator; The Playwright tool is used to scan the Document Object Model (DOM) structure of the target page, and natural language processing technology is used to semantically match the text description of the target interface element in the natural language testing step with the text content and accessibility attributes of the elements in the DOM structure in order to identify and extract the positioning method of the target interface element. The location methods and associated attributes of the successfully extracted target interface elements are incrementally updated to the UI element knowledge base as new entries. Based on the location methods of interface elements retrieved from the UI element knowledge base and the location methods of the extracted target interface elements, and in conjunction with the natural language testing steps, executable test cases are generated.

6. The method according to claim 1, characterized in that, After generating test cases, the method further includes: Based on the ISTQB software testing standards of the international software testing engineer certification body, the natural language testing steps are analyzed to obtain enhanced testing steps that include abnormal paths and boundary values. The generated test cases are enhanced according to the enhanced testing steps to obtain enhanced test cases.

7. The method according to claim 6, characterized in that, The analysis of the natural language testing steps, based on the ISTQB software testing standards (an internationally recognized certification body for software testing engineers), includes: From the historical test case knowledge base, the RAG technology is used to retrieve historical abnormal path test case fragments that are similar to the functions described in the natural language test steps; The retrieved historical abnormal path test case fragments are combined with the natural language testing steps and the ISTQB software testing standard to serve as input context, guiding the generation of the enhanced testing steps that include abnormal paths and boundary values.

8. A test case generation device, characterized in that, include: The receiving unit is used to receive natural language test requirement prompts input by the user; The retrieval unit is used to retrieve function description-related information from the plug-in knowledge base based on the natural language testing requirement prompts and through retrieval enhancement generation RAG technology; wherein, the plug-in knowledge base includes a function description knowledge base and a UI element knowledge base; the function description knowledge base stores function description documents describing the business logic of the function; the function description documents include at least one of a product requirement document and a product prototype annotation description document; the UI element knowledge base stores the positioning methods and element attributes of interface elements; The acquisition unit is used to acquire structured natural language testing steps based on the pre-trained large language model, the natural language testing requirement prompts, and the functional description association information. The generation unit is used to generate test cases based on the natural language testing steps and the location method of the interface elements retrieved from the UI element knowledge base based on the natural language testing requirements.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer-executable instructions that, when executed by a processor, implement the steps of the method described in any one of claims 1 to 7.

Images