Test case generation method and related device

Abstract

A test case generation method, comprising: receiving test requirements of a user; retrieving a knowledge base on the basis of the test requirements to obtain test objects related to the test requirements, wherein the test objects related to the test requirements include test objects corresponding to similar requirements; deriving use case generation assistance information by means of a language model on the basis of at least one of: the test objects related to the test requirements, a requirement document of the test requirements, or associated information, wherein the associated information comprises at least one of associated functions or scenarios, and the use case generation assistance information comprises at least one of: at least one test point or at least one test-related factor; and then, on the basis of the at least one test point or the at least one test-related factor, generating test cases corresponding to the test requirements. In the method, by recommending a test point or a test-related factor on the basis of a knowledge base, and integrating the recommendation of the test point or the test-related factor into a test operation process, quick reference to a historical test design process can be achieved, thereby improving test design efficiency.

Description

A test case generation method and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202510198809.8, filed on February 20, 2025, entitled "A Test Case Generation Method and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of testing technology, and in particular to a test case generation method, a testing system, a computing device cluster, a computer-readable storage medium, and a computer program product. Background Technology

[0003] To ensure the quality of software and hardware, test cases (TCs) are typically developed to test them. For ease of description, this application primarily uses software testing examples. A test case is a set of test inputs, execution conditions, and expected results compiled for a specific objective, in order to test a program path or verify whether a specific requirement is met.

[0004] During the test design phase, testers typically need to comprehensively analyze each test point and related factors such as test factors, data factors, and action factors based on the requirements when designing tests for the test object. They also need to evaluate the completeness of the tests. Test points, also known as test function points, refer to the test checks set for various functions of the software. Taking e-commerce software as an example, test points can include categories such as products, shopping cart, orders, and user reviews. Product test points include: checking whether product images are clear and fully displayed, and whether there are any loading failures; confirming the accuracy of product information; verifying the accuracy of product category navigation and whether it can correctly guide users to find the products they need. Shopping cart test points include: testing the function of adding products to the shopping cart, confirming whether the quantity and total price are correct; checking whether the operations of modifying the quantity of products in the shopping cart and deleting products are normal; verifying whether the shopping cart's state is correctly saved after refreshing different pages and logging in / out.

[0005] This results in test case generation being highly dependent on the testers' experience and understanding of the business, leading to low test design efficiency. Summary of the Invention

[0006] This application provides a test case generation method. This method retrieves test objects related to test requirements from a knowledge base during the test design phase. Based on these test objects, test points or test-related factors are automatically generated using language models such as LLM. This enables the recommendation / generation of test points or test-related factors based on a test experience base. Integrating the recommendation of test points or test-related factors into the test workflow allows for rapid referencing of historical test design processes, improving test design efficiency. This application also provides a corresponding test system, computing device cluster, computer-readable storage medium, and computer program product.

[0007] Firstly, this application provides a test case generation method. This method can be executed by a testing system. The testing system can be a software system, which can be a standalone software system or integrated into other software as a plugin, functional module, component, or app. When the testing system is a software system, it can be provided to the customer as a software package for self-deployment. In some examples, the testing system may include a hardware system; for instance, the testing system may include a cluster of computing devices that supports test case generation. When the computing device cluster is running, it executes the test case generation method of this application.

[0008] In practical implementation, the testing system can receive user test requirements, retrieve the knowledge base based on the test requirements, and obtain test objects related to the test requirements. Test objects related to the test requirements include test objects with similar requirements. A test object is the target or entity targeted by the test. Test objects can include at least one of the following: software or hardware code, components, interfaces, documents, or data. Similar requirements include requirements with a similarity greater than a threshold to the user's test requirements. The testing system can derive test cases and generate auxiliary information based on at least one of the test objects related to the test requirements, the requirement documents of the test requirements, or related information, using a language model. Related information includes at least one of the associated functions or scenarios. Test case generation auxiliary information includes at least one of at least one test point or at least one test-related factor. The testing system can generate test cases corresponding to the test requirements based on at least one test point or at least one test-related factor.

[0009] This method retrieves test objects related to test requirements from a knowledge base during the test design phase. Based on these test objects, test points or test-related factors are automatically generated using a language model. This enables the recommendation / generation of test points or test-related factors based on a test experience base. Integrating these recommendations into the test workflow allows for rapid referencing of historical test design processes, improving test design efficiency.

[0010] In some possible implementations, the testing system can deduce the requirement type of the test requirement using a language model based on the test requirements or related information. The language model can be a large language model, including but not limited to various large-scale language models based on transformers. Then, the testing system can deduce the test objects of the test requirements using the language model based on the requirement type and related information. Next, the testing system can perform enhanced retrieval of the knowledge base based on at least one of the requirement type, the test objects of the test requirements, the requirement document of the test requirements, or related information to obtain the test objects related to the test requirements.

[0011] This method introduces a thought chain mechanism in the process of recommending test points or test-related factors. It progressively analyzes and breaks down the recommendation process for test points and related factors during test design, for example, by deducing the requirement type, deducing the test object (also known as the requirement object), retrieving the test object, and deducing the test points or test-related factors. This reflects the analytical process of test design and enables precise retrieval and recommendation, improving the work efficiency of test designers. Furthermore, by providing the derivation and analysis process, this method avoids abrupt jumps (such as directly outputting results), making it easy for users to understand and thus increasing the adoption rate of tests or test-related factors.

[0012] In some possible implementations, the testing system can also retrieve at least one of the following from a knowledge base: requirement documents or related information for the test requirements, to obtain requirement documents or related information for similar requirements. Accordingly, when deriving auxiliary information for test case generation, the testing system can derive auxiliary information for test cases using a language model based on at least one of the following: test objects related to the test requirements, requirement documents or related information for the test requirements, and requirement documents or related information for similar requirements.

[0013] This method further retrieves requirement documents and related information for similar requirements, and adds these documents and related information to the input of the language model (e.g., prompt), thereby providing more information to the language model and improving its generation performance.

[0014] In some possible implementations, the testing system can also determine test case coverage using mind maps. When a user triggers a regeneration action, the testing system can regenerate test cases corresponding to the test requirements. For example, the testing system can re-derive test points or test-related factors, and based on the re-derived test points and test-related factors, generate test cases corresponding to the test requirements.

[0015] This allows for rapid measurement and evaluation of the overall coverage of the test design during the test design phase, quick interception of missing test cases, early identification of missing coverage scenarios, and ensuring the comprehensiveness of test scenarios.

[0016] In some possible implementations, the testing system can also obtain data from the data source to build the knowledge base, then vectorize the data to generate knowledge vectors, and store the knowledge vectors in the knowledge base.

[0017] This method constructs a knowledge base based on historical data or experience, which can lay the foundation for subsequent enhanced retrieval. This knowledge base can help in the step-by-step analysis and decomposition of the recommendation process for test points and test-related factors.

[0018] In some possible implementations, the knowledge base includes at least one of the following: a requirements knowledge base, a functional knowledge base, a scenario knowledge base, a test point knowledge base, a problem form knowledge base, a document knowledge base, a test factor knowledge base, or a test mode knowledge base. Different knowledge bases can provide different types of information to meet business needs.

[0019] In some possible implementations, the testing system can provide the user with at least one test point or at least one test-related factor. The testing system can then determine at least one of the target test points or target factors selected by the user. Accordingly, the testing system can generate test cases corresponding to the test requirements based on at least one of the target test points or target factors.

[0020] This method improves the quality of generated test cases by providing users with recommended test points and test-related factors, allowing users to select template test points and target factors to generate test cases.

[0021] In some possible implementations, test-related factors include at least one of test factors, data factors, or action factors. Test factors include various factors or conditions used to determine the test scope, test content, and test methods during software testing. Test factors can be abstractions and generalizations of various elements in a software system that may affect its functionality, performance, reliability, etc., and are an important basis for designing test cases. For example, for a mobile application, test factors might include different operating system versions, screen resolution, network environment (such as 4G, 5G, Wi-Fi), and user operation frequency. Data factors include various data-related factors involved in the testing process. For example, when testing the order function of an e-commerce system, data factors might include the quantity of goods, price, discount information, user address, payment method, etc. Action factors include various operations or behaviors performed by the user or system during software operation. For example, when testing a text editing software, action factors include operations such as inputting text, deleting text, copying and pasting, formatting, saving files, and opening files.

[0022] This method can provide sufficient information for test case generation by recommending test factors, data factors, or action factors, thereby improving the quality of test case generation.

[0023] Secondly, this application provides a testing system. The system includes:

[0024] The interaction module is used to receive users' test requests;

[0025] The retrieval enhancement module is used to retrieve the knowledge base according to the test requirements and obtain test objects related to the test requirements. The test objects related to the test requirements include test objects with similar requirements. The test objects include at least one of software and hardware code, components, interfaces, documents or data. The similar requirements include requirements with a similarity to the user's test requirements greater than a threshold.

[0026] The derivation module is used to derive test cases and generate auxiliary information through a language model based on at least one of the test objects related to the test requirements, the requirement documents of the test requirements, or related information. The related information includes at least one of the related functions or scenarios, and the test case generation auxiliary information includes at least one of at least one test point or at least one test-related factor.

[0027] The test case generation module is used to generate test cases corresponding to the test requirements based on the at least one test point or the at least one test-related factor.

[0028] In some possible implementations, the derivation module is further used for:

[0029] Based on the test requirements or the associated information, the requirement type of the test requirements is derived using the language model;

[0030] Based on the requirement type and the associated information, the test object of the test requirement is derived through the language model;

[0031] The retrieval enhancement module is specifically used for:

[0032] Enhanced retrieval of the knowledge base is performed based on at least one of the requirement type, the test object of the test requirement, the requirement document of the test requirement, or related information to obtain the test object related to the test requirement.

[0033] In some possible implementations, the retrieval enhancement module is further used for:

[0034] Search the knowledge base based on at least one of the requirement documents or related information of the test requirements to obtain at least one of the requirement documents or related information of similar requirements;

[0035] The derivation module is specifically used for:

[0036] Auxiliary information is generated by deriving test cases through a language model based on at least one of the test objects related to the test requirements, the requirement documents or related information of the test requirements, and the requirement documents or related information of similar requirements.

[0037] In some possible implementations, the system further includes:

[0038] The coverage estimation module is used to determine the coverage of the test cases through a mind map;

[0039] The test case generation module is used to regenerate the test cases corresponding to the test requirements when the user triggers the regeneration operation.

[0040] In some possible implementations, the system further includes:

[0041] A knowledge base construction module is used to obtain data from a data source for constructing the knowledge base; to vectorize the data to generate knowledge vectors; and to store the knowledge vectors in the knowledge base.

[0042] In some possible implementations, the knowledge base includes at least one of the following: a requirements knowledge base, a functional knowledge base, a scenario knowledge base, a test point knowledge base, a problem sheet knowledge base, a document knowledge base, a test factor knowledge base, or a test mode knowledge base.

[0043] In some possible implementations, the interaction module is also used for:

[0044] Provide the user with at least one test point or at least one test-related factor;

[0045] Determine at least one of the target test points or target factors selected by the user;

[0046] The test case generation module is specifically used for:

[0047] Based on at least one of the target test points or the target factors, generate test cases corresponding to the test requirements.

[0048] In some possible implementations, the test-related factors include at least one of test factors, data factors, or action factors.

[0049] Thirdly, this application provides a computing device cluster. The computing device cluster includes at least one computing device, which includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is used to execute instructions stored in the at least one memory to cause the computing device or the computing device cluster to perform the test case generation method as described in the first aspect or any implementation thereof.

[0050] Fourthly, this application provides a computer-readable storage medium storing instructions that instruct a computing device or a cluster of computing devices to execute the test case generation method described in the first aspect or any implementation thereof.

[0051] Fifthly, this application provides a computer program product containing instructions that, when run on a computing device or a cluster of computing devices, causes the computing device or cluster of computing devices to execute the test case generation method described in the first aspect or any implementation thereof.

[0052] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0053] To more clearly illustrate the technical methods of this application, the accompanying drawings used will be briefly described below.

[0054] Figure 1 is a schematic diagram of the architecture of a test system provided in this application;

[0055] Figure 2 is a flowchart of a test case generation method provided in this application;

[0056] Figure 3 is a flowchart of a test case generation method provided in this application;

[0057] Figure 4 is a schematic diagram of the process of knowledge base construction and test point and test-related factor recommendation provided in this application;

[0058] Figure 5 is a flowchart illustrating a test case generation method provided in this application;

[0059] Figures 6A to 6C are schematic diagrams of the interface for the test case generation process provided in this application;

[0060] Figure 7 is a schematic diagram of the structure of a test system provided in this application;

[0061] Figure 8 is a schematic diagram of the structure of a computing device provided in this application;

[0062] Figure 9 is a schematic diagram of the structure of a computing device cluster provided in this application;

[0063] Figure 10 is a schematic diagram of another computing device cluster provided in this application;

[0064] Figure 11 is a schematic diagram of another computing device cluster provided in this application. Detailed Implementation

[0065] The terms "first" and "second" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0066] First, some technical terms involved in the embodiments of this application will be introduced.

[0067] A test case is a set of test inputs, execution conditions, and expected results designed for a specific purpose, in order to test a program path or verify whether a specific requirement is met. Test cases include software test cases and hardware test cases. Software test cases are used to test the functionality or performance of software, while hardware test cases are used to test the functionality or performance of hardware. For ease of description, this application uses a software test case example.

[0068] A test point, or simply test point, is a specific item or focus that reflects the quality characteristics of software and hardware, such as functionality, performance, compatibility, and security, and requires testing and verification. These items or focuses requiring testing and verification are also called test check items.

[0069] Test points are the foundation of testing work, providing direction for the design of test cases and helping testers to comprehensively and systematically verify software and hardware to ensure that they meet quality requirements.

[0070] To make it easier to understand, the following explanation uses the login function as an example to illustrate the test points and test cases.

[0071] Test points may include: username and password input verification; login button functionality; remember password function; error handling; third-party login. Username and password input verification may include the following test checks: whether a valid username and password can be used to log in successfully; prompts when the username or password is empty; username length exceeding the limit; password length not meeting requirements. Login button functionality may include the following test checks: whether clicking the login button triggers the login operation; response to rapid, consecutive clicks of the login button. The remember password function may include the following test checks: whether the username and password are automatically filled in the next time the page is opened after selecting "remember password"; whether the username and password are not automatically filled in the next time the page is opened after unchecking "remember password". Error handling may include the following test checks: prompts when an incorrect username or password is entered; handling and prompts for login when there is a network error. Third-party login functionality may include the following test checks: whether the third-party login button displays correctly; whether clicking the third-party login button redirects to the corresponding authorization page; whether the user can successfully enter the system after successful third-party login.

[0072] One or more test cases can be designed for each test point. For example, for username and password input validation, the following test cases can be designed:

[0073] Use Case 1: Login with valid username and password

[0074] Test steps: Enter a valid registered username in the username input box, enter the correct password in the password input box, and click the login button.

[0075] Expected result: Successfully log in to the system and be redirected to the system homepage.

[0076] Use Case 2: Username is empty, password is not empty

[0077] Test steps: Leave the username input box blank, enter any characters in the password input box, and click the login button.

[0078] Expected result: A pop-up message will appear stating "Username cannot be empty".

[0079] Use Case 3: Username is not empty, password is empty

[0080] Test steps: Enter any characters in the username input box, leave the password input box blank, and click the login button.

[0081] Expected result: A pop-up message will appear stating "Password cannot be empty".

[0082] Use Case 4: Username length exceeds limit

[0083] Test steps: Enter a string exceeding the maximum length limit in the username input box, enter any characters in the password input box, and click the login button.

[0084] Expected result: A pop-up message will appear stating "Username length cannot exceed [X] characters" (X is the maximum length set by the system).

[0085] Use Case 5: Password length does not meet requirements (too short)

[0086] Test steps: Enter any characters in the username input box, enter a string shorter than the minimum length limit in the password input box, and click the login button.

[0087] Expected result: A pop-up message will appear stating "Password length must be at least [X] characters" (X is the minimum length set by the system).

[0088] With the rise of large language models (LLMs) and other large-scale models, the industry has proposed solutions for automatically generating test cases based on these models. Specifically, this involves first acquiring a large amount of historical data, which can include historical requirement documents, design documents, user manuals, and test cases. This historical data is preprocessed, and then feature extraction is performed through feature engineering. Based on the extracted features, the large model is trained and optimized using deep learning algorithms. The large model can be an artificial intelligence (AI) model with a transformer architecture at its core. It utilizes a self-attention mechanism to handle long-distance dependency problems and is trained to identify key patterns, potential failure points, and specific test scenario descriptions in test cases. During training, the large model's convergence is checked; if it doesn't converge, training and optimization continue until convergence occurs. A converged large model can understand and simulate complex testing environments, thus supporting the automatic generation of test cases. Furthermore, by combining software system parameters and using tools from the Catalon Studio and TestTrack tools, the large model can automatically generate more targeted test cases.

[0089] However, during the test design process, testers lack real-time recommendations from historical experience databases. Often, they need to comprehensively analyze each test point and test factor based on requirements, and assess the completeness of the tests. This approach heavily relies on the testers' experience and understanding of the business. This test analysis method demands that testers have a comprehensive understanding of the business to ensure test case coverage. Consequently, the design threshold for test cases is high, and the efficiency of test design is low.

[0090] In view of this, this application provides a test case generation method. This method can be executed by a testing system. The testing system can be a software system, which can be a standalone software system or integrated into other software in the form of a plugin, functional module, component, or applet. When the testing system is a software system, it can be provided to the customer as a software package for self-deployment. In some examples, the testing system may include a hardware system; for example, the testing system may include a cluster of computing devices that supports test case generation. When the computing device cluster is running, it executes the test case generation method of this application.

[0091] Specifically, the testing system receives user test requirements, searches the knowledge base based on these requirements, and obtains test objects related to the test requirements. Test objects related to the test requirements include test objects with similar requirements. A test object is the target or entity being tested, including software and hardware code (such as software system code), components (such as modules, classes, or functions), interfaces (including but not limited to interfaces between internal software modules, interfaces between software and external systems or devices), documents (such as requirement documents, design documents, and user manuals), and data (data processed and stored during operation, such as customer account information and transaction records maintained by a bank system). A test object may include multiple test function points, which are refinements of the test object's functionality. Similar requirements are those with a similarity to the user's test requirements greater than a threshold. The testing system can then derive test cases and generate auxiliary information based on at least one of the test objects related to the test requirements, the requirement documents for the test requirements, or related information, using a language model. Related information includes at least one of the associated functions or scenarios, and auxiliary information for test case generation includes at least one test point or at least one test-related factor. The testing system can generate test cases corresponding to the test requirements based on at least one test point or at least one test-related factor.

[0092] This method retrieves test objects related to test requirements from a knowledge base during the test design phase. Based on these test objects, test points or test-related factors are automatically generated using a language model. This enables the recommendation / generation of test points or test-related factors based on a test experience base. Integrating these recommendations into the test workflow allows for rapid referencing of historical test design processes, improving test design efficiency.

[0093] To make the technical solution of this application clearer and easier to understand, the system architecture of this application will be described below with reference to the accompanying drawings.

[0094] Referring to Figure 1, which shows a schematic diagram of the architecture of a testing system, the testing system 10 includes a requirements design platform 100 and a testing platform 200. Currently, the requirements design phase and the test design phase are separate. This application introduces an auxiliary test design agent into the requirements design platform 100, thereby integrating the test design phase and the requirements design phase, allowing systems engineers (SE), developers, and testers to collaboratively complete the test design.

[0095] The requirements design platform 100 is used to assist in designing test requirements. The requirements design platform 100 includes a test desktop 102, which can include at least one of the following: feature tree, architecture design, knowledge base design, bridging configuration, work items (requirements), or defect library. A feature tree is a way to organize and display the characteristics of a software system in a tree structure. The feature tree arranges various characteristics of the software system according to hierarchical relationships, progressively subdividing from the top-level main characteristics to the lower-level sub-characteristics, similar to a directory structure, clearly presenting the overall functionality of the software system. The feature tree helps the project team fully understand the software system requirements and clarify the relationships and dependencies between various characteristics. Architecture design is the design of the overall structure of the software system, component division, inter-component interactions, and the relationship between the system and the external environment. It determines key aspects such as the software's organizational structure, operating mode, performance, and scalability. Knowledge base design is the design of a system for storing, organizing, and managing knowledge. Bridging configuration typically refers to establishing connections and setting parameters between different networks, systems, components, or protocols to achieve communication and interaction between them. In this application, bridging configuration is used for synchronous space mapping, such as establishing a mapping relationship between test requirements and test design tasks. A work item is an abstract description of a single task or activity that needs to be completed in project management. It includes detailed information such as title, description, responsible person, priority, and deadline. Basic requirement information in a work item can be synchronized to the requirements knowledge base. A defect repository is a database or storage system used to store all defect information discovered during the development, testing, and operation of a software project.

[0096] The requirements design platform 100 also includes a test design assisting agent 104, used to assist in designing test cases. The test design assisting agent 104 includes retrieval enhancement, domain document slicing, domain prompts, and a rich text processing component. Retrieval enhancement optimizes the output of the language model through Retrieval Augmented Generation (RAG). Domain document slicing segments domain documents to build a knowledge base. The rich text processing component processes rich text format documents, such as requirements documents including diagrams. Domain prompts provide relevant domain-specific prompts to enhance the requirements information.

[0097] The testing platform 200 is used to receive user test requirements, such as those received from the requirements design platform 100. It then searches a knowledge base based on these requirements to obtain test objects related to the test requirements. Test objects related to the test requirements include test objects with similar requirements. The testing platform 200 is also used to derive test cases and generate auxiliary information using a language model based on at least one of the test objects related to the test requirements, the requirements document for the test requirements, or related information. The related information includes at least one of the associated functions or scenarios, and the test case generation auxiliary information includes at least one test point or at least one test-related factor. Based on at least one test point or at least one test-related factor, test cases corresponding to the test requirements are generated.

[0098] Test-related factors can include test factors, data factors, or action factors. Test factors include various factors or conditions used to determine the test scope, test content, and test methods during software testing. Test factors can be abstractions and generalizations of various elements in a software system that may affect its functionality, performance, reliability, etc., and are an important basis for designing test cases. For example, for a mobile application, test factors might include different operating system versions, screen resolution, network environment (such as 4G, 5G, Wi-Fi), and user operation frequency. Data factors include various data-related factors involved in the testing process. For example, when testing the order function of an e-commerce system, data factors might include the quantity of goods, price, discount information, user address, payment method, etc. Action factors include various operations or behaviors performed by the user or system during software operation. For example, when testing a text editing software, action factors include operations such as inputting text, deleting text, copying and pasting, formatting, saving files, and opening files.

[0099] The interaction flow of the testing platform 200 is as follows: The testing platform 200 provides a task management interface, where users can bind test requirements (e.g., original requirement issue requests, denoted as IR) to test design tasks according to business needs, thereby ensuring that the test design accurately corresponds to business requirements. The test design tasks are organized and stored using a test design mind map 202 (or simply a mind map). A mind map is a tool that uses hierarchical structures and branches to represent the thought process. The test design mind map 202 in the testing platform 200 automatically obtains associated sub-requirements through the requirement IR bound to the task. It retrieves requirement documents or related information through the requirement information of the test requirements and sub-requirements, where related information can include associated functions and scenarios. Furthermore, the testing platform 200 also supports the integration of retrieved requirement documents and related information. By associating and integrating scenarios, functions, and requirement documents, the testing platform 200 can further retrieve similar scenarios, functions, and requirement documents, ensuring the comprehensiveness of the test design.

[0100] The test agent 204 in the test platform 200 takes at least one of the following as input: test objects related to the test requirements, requirement documents for the test requirements, or related information. It then uses a language model to deduce test cases and generate auxiliary information. For example, the test agent can take related scenarios, functions, requirement documents, similar requirements, and other data as input, and call a large model algorithm service (such as LLM) to recommend relevant test factors and test points.

[0101] When recommending test factors or test points, the test agent 204 can also search the knowledge base (such as the requirement knowledge base 206) and add the search results to the prompts input to the large model. A prompt is a text command input when interacting with a large model (such as an LLM), guiding the large model to generate output of a specific type, style, or content. Specifically, the test agent 204 can request prompts and search the knowledge base, then add the search results to the requested prompts. The test agent 204 can also obtain a prompt template for the test design task and generate prompts that match the test design task based on that template.

[0102] Specifically, test agent 204 can retrieve test objects related to the test requirements from the knowledge base. Test objects related to the test requirements include test objects with similar requirements. Test agent 204 can derive test cases and generate auxiliary information using a language model based on at least one of the test objects related to the test requirements, the requirement document of the test requirements, or related information. Related information includes at least one of the associated functions or scenarios, and the test case generation auxiliary information includes at least one of at least one test point or at least one test-related factor. Correspondingly, test platform 200 (e.g., test agent 204 in test platform 200) can generate test cases corresponding to the test requirements based on at least one test point or at least one test-related factor.

[0103] In some possible implementations, the testing platform 200 may also provide the user with at least one test point or at least one test-related factor, determine at least one of the target test points or target factors selected by the user, and generate test cases corresponding to the test requirements based on at least one of the target test points or target factors. Specifically, the testing platform 200 may provide the user with at least one test point or at least one test-related factor through a test design mind map 202 for the user to select.

[0104] Furthermore, the testing platform 200 can also determine test case coverage using mind maps (such as test design mind map 202). Specifically, the test design mind map 202 automatically calculates the test case coverage corresponding to the current testing requirements after the user selects and references test-related factors (such as test factors, data factors, and action factors) and test points. The testing platform 200 can estimate test case coverage based on the user-selected test-related factors and test points, as well as test-related factors and test points for similar requirements.

[0105] Furthermore, users can decide whether to trigger the operation of re-deriving test case generation auxiliary information (or simply regeneration operation) based on the test case coverage. The test platform 200 can also respond to the user-triggered regeneration operation by re-deriving test case generation auxiliary information through a language model based on at least one of the test objects related to the test requirements, the test requirement document, or associated information. The test platform 200 can also provide the user with the re-derived test case generation auxiliary information, regenerating test cases corresponding to the test requirements based on the test-related factors and test points selected by the user.

[0106] In this application, the testing system 10 integrates recommendations from a testing experience base (test points, test-related factors) into the test design process during the test design phase, automatically generating test cases using the recommended test points and factors, thereby improving test design efficiency. Furthermore, based on chain of thought (COT) and retrieval enhancement technology, the testing system 10 progressively analyzes and decomposes the recommendation process of requirement-related test points and factors during the test design process, realizing the test design analysis process and performing accurate retrieval and recommendations, thus improving the work efficiency of test designers. In addition, this application, based on the generalization capability of a large model, can be used for cross-product, cross-application, and cross-class test design analysis and test case generation, and achieves early identification of test design omissions during the test design phase, ensuring test case coverage.

[0107] Based on the test system 10 shown in Figure 1, this application provides a test case generation method. The test case generation method provided by the embodiments of this application will be described below with reference to the accompanying drawings.

[0108] Referring to Figure 2, a flowchart of a test case generation method is shown. The method includes the following steps:

[0109] S202, Test System 10 receives the user's test request.

[0110] Test requirements can be descriptions of the requirements for test design tasks. Test design tasks can be tasks performed during the test design phase, including but not limited to designing test cases. Test requirements can be descriptions based on natural language, or on programming languages ​​or structured languages.

[0111] Taking an e-commerce system as an example, testing requirements may include: "Users can register by mobile phone number or email address and can only place an order after completing identity verification"; "Users can filter products by category, price or rating, or search for specific products through the search bar"; "Users can add products to their shopping cart, update the quantity, and checkout payment".

[0112] S204. Test system 10 retrieves test objects related to the test requirements from the knowledge base based on the test requirements.

[0113] Test objects related to testing requirements include test objects with similar requirements. Similar requirements can be test requirements in the knowledge base that are similar to the user's test requirements, such as test requirements in the requirements knowledge base whose similarity to the user's test requirements reaches a threshold. Test objects include the targets that need to be tested during the software testing process. Test objects can include at least one of the following: software and hardware code, components, interfaces, documentation, or data. Taking a software system as an example, test objects cover multiple levels, from the software system itself (such as the software system's code), software components within the software system (such as modules in the software), interfaces (such as interfaces between modules), to related documentation and data.

[0114] Software system-level testing includes at least one of the following: system functionality, system performance or compatibility, and system security. System functionality testing verifies whether the software system can perform its intended functions. For example, in an e-commerce system, functionality testing primarily checks whether functions such as product display, shopping cart management, order processing, and payment function correctly. System performance testing assesses the software system's performance under different conditions, such as response time, throughput, and concurrent processing capabilities. Using an online game as an example, performance testing measures the response speed and stability when a large number of players are online simultaneously. System compatibility testing examines the software system's operation in different environments, including different operating systems, browsers, and devices (such as mobile phones, tablets, and computers). System security testing checks whether the software system possesses sufficient security mechanisms, such as user authentication, authorization management, data encryption, and vulnerability prevention. For example, for a banking system, security testing requires verifying whether the security of user account information and transaction data is rigorously protected.

[0115] Software component-level testing includes at least one of module functionality or module interfaces. Module functionality testing is used to test individual modules within a software system to ensure their correct functionality. For example, in large enterprise management software, module functionality testing is used to perform functional tests on modules such as human resources and financial management. Module interface testing is used to check whether the interfaces between modules can communicate and exchange data normally. Using large enterprise management software as an example, module interface testing is used to test whether the data transmission format and calling rules between different subsystems (such as modules) meet the requirements.

[0116] Software documentation-level testing includes at least one of the following: requirements document, design document, and user manual. Requirements document testing verifies the completeness, accuracy, and consistency of the requirements document, ensuring it accurately reflects the user's testing needs. In practice, requirements document testing can be achieved by checking the clarity of the functional descriptions and identifying any omissions or contradictions. Design document testing verifies the rationality of the design document and its ability to guide the development and implementation of the software system. In practice, design document testing may include evaluating the scalability and stability of the architectural design. User manual testing verifies the accuracy and understandability of the user manual's content and its ability to help users correctly use the software system. In practice, user manual testing may include checking the detail of the operation steps and the appropriateness of the examples.

[0117] Data-level testing includes at least one of the following: data accuracy, data integrity, or data consistency. Data accuracy testing verifies whether the data content is accurate, such as whether customer information or product data in a database matches the actual situation. Data integrity testing checks whether the data is complete and free from missing or corrupted data. For example, ensuring data integrity during data migration. Data consistency testing ensures data consistency between different data sources or systems. For example, ensuring data consistency when synchronizing data between multiple databases.

[0118] The knowledge base includes at least one of the following: a requirements knowledge base, a functional knowledge base, a scenario knowledge base, a test point knowledge base, a problem sheet knowledge base, a document knowledge base, a test factor knowledge base, or a test mode knowledge base. In specific implementation, the test system 10 can retrieve similar requirements from the requirements knowledge base based on the user's test needs, and then the test system 10 can obtain test objects with similar requirements.

[0119] S206. The test system 10 generates auxiliary information by deriving test cases through a language model based on at least one of the test objects, test requirement documents, or related information related to the test requirements.

[0120] The associated information includes at least one of the associated functions or scenarios. The associated functions can be obtained from the function knowledge base, and the associated scenarios can be obtained from the scenario knowledge base. The requirement document for the test requirements can be obtained from the document knowledge base. For example, test system 10 can retrieve the requirement document for the test requirements from the document knowledge base based on the user's test requirements.

[0121] The testing system can construct cue words based on at least one of the following: test objects related to the test requirements, requirement documents for the test requirements, or related information. Based on these cue words, it can derive test cases and generate auxiliary information using language models such as LLM. Test case generation auxiliary information is used to generate test cases and includes at least one of the following: at least one test point or at least one test-related factor.

[0122] Test-related factors include at least one of test factors, data factors, or action factors. Test factors encompass various factors or conditions used to determine the test scope, content, and methods during software testing. Test factors can be abstractions and generalizations of various elements in a software system that may affect its functionality, performance, reliability, etc. For example, for a mobile application, test factors might include different operating system versions, screen resolution, network environment, and user operation frequency. Data factors include various data-related factors involved in the testing process. For example, for an e-commerce system, data factors might include product quantity, price, discount information, user address, payment method, etc. Action factors include various operations or behaviors performed by the user or system during software operation. For example, for text editing software, action factors include operations such as inputting text, deleting text, saving files, and opening files.

[0123] In some possible implementations, the test system 10 can also retrieve at least one of the following from a knowledge base: requirement documents or related information of the test requirements, to obtain at least one of the following requirement documents or related information of similar requirements. Accordingly, the test system 10 can derive use cases and generate auxiliary information based on at least one of the following: test objects related to the test requirements, requirement documents or related information of the test requirements, and requirement documents or related information of similar requirements, through a language model.

[0124] S208. The test system 10 generates test cases corresponding to the test requirements based on at least one test point or at least one test-related factor.

[0125] The testing system 10 can construct cue words based on at least one test point or at least one test-related factor, and derive test cases corresponding to the test requirements based on the cue words through a language model. It should be noted that the language model used to derive the test cases and the language model used to generate auxiliary information for deriving the test cases can be the same language model or different language models, for example, language models trained using different corpora.

[0126] In some possible implementations, the testing system 10 may provide the user with at least one test point or at least one test-related factor, and then determine at least one of the target test point or target factor selected by the user. Next, the testing system 10 can generate test cases corresponding to the test requirements based on at least one of the target test point or target factor.

[0127] Based on the above description, this application provides a test case generation method. This method, targeting user testing needs, automatically recommends test-related factors or test points as auxiliary information for test case generation by retrieving knowledge bases and using language model reasoning. This information is then selected by test designers, and test cases are automatically generated based on the user-selected test-related factors or test points. This integrates the recommendations of test-related factors and test points into the test design workflow, improving test design efficiency and meeting the needs of test designers' test design activities.

[0128] Considering that the existing solutions directly output test cases based on the user's testing requirements without reflecting the derivation and analysis process, and the jumps are too large, making it difficult for users to understand, this application uses COT to stepwise analyze and decompose the recommendation process of requirement-related test points and factors (such as test factors, data factors, or action factors) in the test design process. This realizes the analysis process of test design and enables accurate retrieval and recommendation, thereby improving the work efficiency of test designers.

[0129] Specifically, the testing system 10 can deduce the requirement type of the test requirement using a language model based on the test requirements or related information. Requirement types can include categories categorized according to different dimensions. In some examples, requirement types can include functional, performance, or security categories. In other examples, requirement types can include user requirements, business or system requirements, project requirements, or product requirements. Then, the testing system 10 can deduce the test objects of the test requirements using a language model based on the requirement type and related information. Next, the testing system 10 can perform enhanced retrieval of the knowledge base based on at least one of the requirement type, the test objects of the test requirements, the requirement document of the test requirements, or related information to obtain test objects related to the test requirements. Accordingly, the testing system 10 can deduce and generate the test points or test-related factors required by the user based on the aforementioned test objects, so as to generate test cases by referencing these test points or test-related factors.

[0130] The test case generation method of this application will be described in detail below with reference to the embodiments.

[0131] Referring to Figure 3, a flowchart of a test case generation method is shown. This method includes the following steps:

[0132] S302, Test System 10 receives the user's test request.

[0133] S304. Test system 10 obtains relevant information based on the user's test requirements.

[0134] The associated information includes related functions or scenarios. The testing system 10 can retrieve the knowledge base according to testing requirements, such as retrieving the function knowledge base or scenario knowledge base, to obtain the associated information.

[0135] Furthermore, the testing system 10 can also obtain sub-requirement information based on the testing requirements. Sub-requirements can be requirements obtained by breaking down the user's testing requirements, such as lower-level requirements. Then, the knowledge base is searched based on the sub-requirement information to obtain related information.

[0136] S306, Test System 10 derives the requirement type of the test requirement through a language model based on the test requirements or related information.

[0137] Specifically, the testing system 10 can construct prompt words based on the test requirements or related information, using prompt templates for inferring the requirement type, and then input the prompts into a language model for reasoning, thereby inferring the requirement type of the test requirements.

[0138] S308, Test System 10 derives the test objects of the test requirements through a language model based on the requirement type and related information.

[0139] Specifically, the testing system 10 can construct prompt words based on the requirement type and related information, using prompt templates for inferring the test object, and then input the prompt words into a language model for reasoning, thereby inferring the test object of the test requirement.

[0140] S310, Test System 10 retrieves test objects related to the test requirements from the knowledge base based on at least one of the test requirements, the type of test requirements, the test objects, and related information.

[0141] Test objects related to test requirements include test objects with similar requirements. Specifically, test system 10 can first search for similar requirements of the test requirements, and then search the knowledge base based on at least one of the following: test requirements, test requirement type, test objects, related information, and similar requirements, to obtain test objects related to the test requirements.

[0142] S312, the test system 10 classifies the retrieved test objects, and then, based on at least one of the test objects, test requirements documents, or related information, derives at least one test point or at least one test-related factor through language modeling.

[0143] Specifically, the testing system 10 can classify the retrieved test objects and then derive corresponding test points or test-related factors according to the classified test objects. The process of deriving test points or test-related factors is similar to deriving requirement types and test objects. The testing system can construct prompt words based on at least one of the requirement documents or related information of the test objects and test requirements, using prompt templates for deriving test points or test-related factors. Then, the prompt words are input into a language model for reasoning, thereby deriving and generating test points or test-related factors.

[0144] S314, The test system 10 provides the user with at least one test point or at least one test-related factor.

[0145] S316, Test system 10 determines at least one of the target test points or target factors selected by the user.

[0146] S318. The test system 10 generates test cases corresponding to the test requirements based on at least one of the target test points or target factors.

[0147] The specific implementations of S314 to S318 can be found in the description above, and will not be repeated here.

[0148] The above scheme recommends test points or test-related factors through three derivations and several retrievals. The retrieval process relies on the construction of a knowledge base. Specifically, the test system 10 can obtain data from a data source to construct the knowledge base, vectorize the data to generate knowledge vectors, and then store the knowledge vectors in the knowledge base.

[0149] To facilitate understanding, a detailed explanation is provided below with reference to the accompanying drawings.

[0150] Referring to Figure 4, which illustrates a flowchart of knowledge base construction and test point / test-related factor recommendation, the data source can include at least one of design documents, product manuals, or requirement information. The testing system 10 can access the data source to obtain data for building the knowledge base. Considering data quality, the testing system 10 can perform data cleaning on the acquired data. Furthermore, the testing system 10 can perform a security scan on the acquired data to ensure data security. Next, the testing system 10 can segment the documents to obtain multiple document blocks. The testing system 10 can vectorize the document blocks, for example, by embedding them, to construct knowledge vectors. Then, the testing system 10 can store the knowledge vectors in the knowledge base. It should be noted that the knowledge base can include both structured and unstructured data.

[0151] During the test design phase, the test system 10 receives the user's test requirements, obtains a prompt template, constructs prompt words, and inputs these prompt words into an LLM (Limited Language Management) to deduce the requirement type, such as functional, performance, or security. Then, based on the requirement type or the associated information of the test requirements, the test system 10 uses the LLM to deduce the test objects for the test requirements. Next, the test system 10 searches the knowledge base based on the deduced test objects to obtain test objects related to the test requirements. Specifically, the test system 10 can call a data dependency service based on the requirement information of the test requirements to obtain search parameters, and then search the knowledge base according to the search parameters (i.e., execute the search enhancement process in the RAG). The test system 10 can classify the retrieved test objects. In this example, the test objects include functional test objects, which can be further classified into boundary values, equivalence classes, or outliers. The test system 10 can perform a third deduction using the LLM for the classified test objects to generate test points or test-related factors. Furthermore, the test system 10 can perform post-processing on the test points or test-related factors to facilitate user understanding. The testing system 10 can output post-processed test points or test-related factors for the user to select.

[0152] To make the technical solution of this application clearer and easier to understand, the test case generation method of this application will be introduced below in combination with specific scenarios.

[0153] Referring to Figure 5, which illustrates a flowchart of a test case generation method, the data sources in this scenario include, but are not limited to, big data analytics services, open-source communities, and asset repositories. The asset repository may include at least one of a test factor library or a pattern library.

[0154] The testing system 10 can access data during the data preprocessing stage, and then perform data cleaning, security scanning, and quality assessment on the data obtained from the accessed data source. The testing system 10 can chunk and embed high-quality data. It can then construct an index based on the vectorized knowledge vectors, such as a best matching 25 (BM25) index. BM25 is a ranking function based on term frequency and inverse document frequency, used to measure the relevance between a document and a query. Next, the testing system 10 can store the knowledge vectors in a database. It should be noted that the knowledge database can include at least one of the following: a requirement knowledge database, a functional knowledge database, a scenario knowledge database, a test point knowledge database, a problem sheet knowledge database, a document knowledge database, a test factor knowledge database, or a test mode knowledge database. The testing system 10 can store the knowledge vectors in the corresponding type of knowledge database based on their type.

[0155] During the test design phase, users can input test requirements through the test desktop and trigger the auxiliary test design agent to perform context acquisition and knowledge acquisition through a window application (widget) in the test agent. The auxiliary test design agent includes a context acquisition module, a document classification text block retrieval tool, a similar requirement / test point retrieval tool, and a test point generator. The auxiliary test design agent can obtain the context of the test requirements through the context acquisition module and retrieve related information, such as the scenarios and functions associated with the test requirements, through the document classification text block retrieval tool. The auxiliary test design agent can request prompt templates, construct prompt words, and then call an inference service (such as Infer Hub) to input the prompt words into a language model, for example, the base model Code Llama, for inference. Test points are generated through multiple inferences, and at least one generated test point is displayed through the test point generation widget. Users can choose from the displayed test points to generate test cases. It should be noted that the test agent can also perform test case checks to calculate test case coverage, which is displayed through the test case check widget. Accordingly, users can decide whether to trigger test point regeneration based on the displayed coverage. Furthermore, the test design agent may also include a test factor generator, which can be used to derive test-related factors using a language model. The test agent may also include a factor generation widget, which can display the derived test-related factors. Users can select test-related factors from at least one of the displayed test-related factors to assist in generating test cases.

[0156] Next, from an interactive perspective, the test case generation method of this application will be introduced.

[0157] Refer to the interface diagrams of the test case generation process shown in Figures 6A to 6C. As shown in Figure 6A, the user selects a project and switches to the test design application. In the interface of the test design application, the user selects a node, which in this example is test task inheritance. Right-clicking on "AI Recommendation" and then "Test Point Design" triggers test point generation or test point recommendation.

[0158] The test agent can obtain requirement information based on the test requirements, including related information, requirement documents, and test points for similar requirements. As shown in Figure 6A, the test agent can display the requirement information, related information, requirement documents, and test points for similar requirements through a window application. Based on the test requirements, related information, requirement documents, and test points for similar requirements, the test agent can deduce and generate test points, displaying them to the user and recommending them for selection. When the user selects and adopts a test point, the test agent can generate test cases based on the adopted test point.

[0159] The window application can also display a detailed process for analyzing related functions and scenarios (affected functions and scenarios), searching design documents, and referencing test points for similar requirements. Taking the search for design documents as an example, as shown in Figure 6B, this application supports selecting chapters of the design document for test design of different granular requirements, and also supports modifying the content to generate test points. Next, as shown in Figure 6C, the test agent can display at least one test point generated for the test requirement or a sub-requirement of the test requirement. In the example in Figure 6C, for the forced shutdown / startup menu optimization, 12 test points are generated. The user selects 10 of these test points and adopts them. Based on the user's adopted test points, the test agent generates the test points for the test requirement in the mind map.

[0160] This method analyzes the recommendation and generation process of test points and test-related factors step by step by constructing a knowledge base, and integrates the knowledge base-based recommendations of test points and test-related factors into the test design process, realizing a complete test design analysis process, achieving accurate retrieval and recommendation, and improving the efficiency of test design work.

[0161] Based on the aforementioned test case generation method, this application also provides a test system 10. The structure of the test system 10 of this application will be described below with reference to the accompanying drawings.

[0162] Referring to Figure 7, which shows a schematic diagram of a test system 10, the test system 10 includes:

[0163] Interaction module 702 is used to receive user test requests;

[0164] The retrieval enhancement module 704 is used to retrieve the knowledge base according to the test requirements and obtain test objects related to the test requirements. The test objects related to the test requirements include test objects with similar requirements. The test objects include at least one of software and hardware code, components, interfaces, documents or data. The similar requirements include requirements with a similarity to the user's test requirements greater than a threshold.

[0165] The derivation module 706 is used to derive test cases and generate auxiliary information through a language model based on at least one of the test objects related to the test requirements, the requirement documents of the test requirements, or related information. The related information includes at least one of the related functions or scenarios, and the test case generation auxiliary information includes at least one of at least one test point or at least one test-related factor.

[0166] The test case generation module 708 is used to generate test cases corresponding to the test requirements based on the at least one test point or the at least one test-related factor.

[0167] For example, the interaction module 702, the retrieval enhancement module 704, the derivation module 706, and the use case generation module 708 described above can be implemented in hardware or in software.

[0168] When implemented through software, the interaction module 702, retrieval enhancement module 704, derivation module 706, and test case generation module 708 can be applications running on computing devices. For example, the interaction module 702 can be the test desktop in the aforementioned embodiments. These applications can also be virtualized and provided to users as virtualization services. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, or container services. Specifically, a VM service can be a service that uses virtualization technology to create a virtual machine (VM) resource pool on multiple physical hosts to provide VMs for users to use on demand. A BMS service is a service that uses virtualization technology to create a BMS resource pool on multiple physical hosts to provide BMS for users to use on demand. A container service is a service that uses virtualization technology to create a container resource pool on multiple physical hosts to provide containers for users to use on demand. A VM is a simulated virtual computer, that is, a logical computer. A BMS is a scalable, high-performance computing service with computing performance indistinguishable from traditional physical machines and features secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service, and container service mentioned above are merely specific examples. In practical applications, virtualization services can also include other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0169] When implemented in hardware, the interaction module 702, retrieval enhancement module 704, derivation module 706, and use case generation module 708 may include at least one computing device, such as a server. Alternatively, the interaction module 702, retrieval enhancement module 704, derivation module 706, and use case generation module 708 may also be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0170] In some possible implementations, the derivation module 706 is further configured to:

[0171] Based on the test requirements or the associated information, the requirement type of the test requirements is derived using the language model;

[0172] Based on the requirement type and the associated information, the test object of the test requirement is derived through the language model;

[0173] The retrieval enhancement module is specifically used for:

[0174] Enhanced retrieval of the knowledge base is performed based on at least one of the requirement type, the test object of the test requirement, the requirement document of the test requirement, or related information to obtain the test object related to the test requirement.

[0175] In some possible implementations, the retrieval enhancement module 704 is further configured to:

[0176] Search the knowledge base based on at least one of the requirement documents or related information of the test requirements to obtain at least one of the requirement documents or related information of similar requirements;

[0177] The derivation module 706 is specifically used for:

[0178] Auxiliary information is generated by deriving test cases through a language model based on at least one of the test objects related to the test requirements, the requirement documents or related information of the test requirements, and the requirement documents or related information of similar requirements.

[0179] In some possible implementations, the system 10 further includes:

[0180] Coverage estimation module 707 is used to determine the coverage of the test cases through a mind map;

[0181] The test case generation module 708 is used to regenerate the test cases corresponding to the test requirements when the user triggers the regeneration operation.

[0182] In some possible implementations, the system 10 further includes:

[0183] The knowledge base construction module 709 is used to obtain data from the data source for constructing the knowledge base; to vectorize the data to generate knowledge vectors; and to store the knowledge vectors in the knowledge base.

[0184] In some possible implementations, the knowledge base includes at least one of the following: a requirements knowledge base, a functional knowledge base, a scenario knowledge base, a test point knowledge base, a problem sheet knowledge base, a document knowledge base, a test factor knowledge base, or a test mode knowledge base.

[0185] In some possible implementations, the interaction module 702 is further used for:

[0186] Provide the user with at least one test point or at least one test-related factor;

[0187] Determine at least one of the target test points or target factors selected by the user;

[0188] The test case generation module 708 is specifically used for:

[0189] Based on at least one of the target test points or the target factors, generate test cases corresponding to the test requirements.

[0190] In some possible implementations, the test-related factors include at least one of test factors, data factors, or action factors.

[0191] This application also provides a computing device 800. As shown in FIG8, the computing device 800 includes: a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate with each other via the bus 802. The computing device 800 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 800.

[0192] Bus 802 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 8, but this does not imply that there is only one bus or one type of bus. Bus 802 can include pathways for transmitting information between various components of computing device 800 (e.g., memory 806, processor 804, communication interface 808).

[0193] Processor 804 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0194] The memory 806 may include volatile memory, such as random access memory (RAM). The memory 806 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD). The memory 806 stores executable program code, which the processor 804 executes to implement the aforementioned test case generation method. Specifically, the memory 806 stores instructions for the test system 10 to execute the test case generation method.

[0195] The communication interface 808 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 800 and other devices or communication networks.

[0196] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0197] As shown in Figure 9, the computing device cluster includes at least one computing device 800. The memory 806 of one or more computing devices 800 in the computing device cluster may store the same instructions of the test system 10 for executing the test case generation method.

[0198] In some possible implementations, one or more computing devices 800 in the computing device cluster can also be used to execute some of the instructions used by the test system 10 to execute the test case generation method. In other words, a combination of one or more computing devices 800 can jointly execute the instructions used by the test system 10 to execute the test case generation method.

[0199] It should be noted that the memories 806 in different computing devices 800 within the computing device cluster can store different instructions for executing certain functions of the test system 10. For example, the memory 806 can store instructions for implementing the functions of the interaction module 702, the retrieval enhancement module 704, the inference module 706, and the test case generation module 708. Furthermore, the memory 806 can also store instructions for implementing the functions of the coverage estimation module 707 and the knowledge base construction module 709.

[0200] Figure 10 illustrates one possible implementation. As shown in Figure 10, two computing devices 800A and 800B are connected via a communication interface 808. The memory in computing device 800A stores instructions for executing the functions of the interaction module 702 and the retrieval enhancement module 704. The memory in computing device 800B stores instructions for executing the functions of the derivation module 706 and the test case generation module 708. Furthermore, the memory in computing device 800A may also store instructions for implementing the functions of the knowledge base construction module 709, and the memory in computing device 800B may also store instructions for implementing the functions of the coverage estimation module 707. In other words, the memory 806 of computing devices 800A and 800B jointly stores the instructions used by the test system 10 to execute the test case generation method.

[0201] The connection method between the computing device clusters shown in Figure 10 can be considered because the test case generation method provided in this application requires a lot of computing resources to derive test points or test-related factors, and to generate test cases based on the test points or test-related factors. Therefore, it is considered to dedicate the functions implemented by the derivation module 706 and the test case generation module 708 to independent computing devices. For example, the functions implemented by the interaction module 702 and the retrieval enhancement module 704 are executed by computing device 800A, and the functions implemented by the derivation module 706 and the test case generation module 708 are executed by computing device 800B.

[0202] It should be understood that the functions of computing device 800A shown in Figure 10 can also be performed by multiple computing devices 800. Similarly, the functions of computing device 800B can also be performed by multiple computing devices 800.

[0203] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 11 illustrates one possible implementation. As shown in Figure 11, two computing devices 800C and 800D are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 806 in computing device 800C stores instructions for executing the functions of the interaction module 702 and the retrieval enhancement module 704. Simultaneously, the memory 806 in computing device 800D stores instructions for executing the functions of the inference module 706 and the use case generation module 708. Further, the memory in computing device 800C may also store instructions for implementing the functions of the knowledge base construction module 709, and the memory in computing device 800D may also store instructions for implementing the functions of the coverage estimation module 707.

[0204] The connection method between the computing device clusters shown in Figure 11 can be considered as follows: taking into account that the test case generation method provided in this application requires a large amount of computing resources to derive test points or test-related factors, and to generate test cases based on test points or test-related factors, it is considered that the functions implemented by the interaction module 702 and the retrieval enhancement module 704 are executed by independent computing devices, such as computing device 800C, and the functions implemented by the remaining subsystems, such as the derivation module 706 and the test case generation module 708, are executed by computing device 800D.

[0205] It should be understood that the functions of the computing device 800C shown in Figure 11 can also be performed by multiple computing devices 800. Similarly, the functions of the computing device 800D can also be performed by multiple computing devices 800.

[0206] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the test case generation method described above applied to the test system 10.

[0207] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the above-described test case generation method.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A test case generation method characterized by, The method comprises: receiving a test requirement of a user; retrieving a knowledge base according to the test requirement to obtain a test object related to the test requirement, the test object related to the test requirement comprising a test object of a similar requirement, the test object comprising at least one of software and hardware code, a component, an interface, a document or data, and the similar requirement comprising a requirement with a similarity to the test requirement of the user greater than a threshold value; deriving, by a language model, case generation auxiliary information according to at least one of the test object related to the test requirement, a requirement document of the test requirement or associated information, the associated information comprising at least one of associated functions or scenarios, and the case generation auxiliary information comprising at least one of at least one test point or at least one test-related factor; generating a test case corresponding to the test requirement according to the at least one test point or the at least one test-related factor.

2. The method of claim 1, wherein, The retrieving of the knowledge base according to the test requirement to obtain the test object related to the test requirement comprises: deriving, by the language model, a requirement type of the test requirement according to the test requirement or the associated information; deriving, by the language model, a test object of the test requirement according to the requirement type and the associated information; performing enhanced retrieval on the knowledge base according to at least one of the requirement type, the test object of the test requirement, a requirement document of the test requirement or the associated information to obtain the test object related to the test requirement.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: retrieving, by the language model, at least one of a requirement document or associated information of the similar requirement according to at least one of the requirement document or the associated information of the test requirement; The deriving of the case generation auxiliary information by the language model according to at least one of the test object related to the test requirement, the requirement document of the test requirement or the associated information comprises: deriving, by the language model, the case generation auxiliary information according to at least one of the test object related to the test requirement, the requirement document of the test requirement, the associated information and the requirement document or the associated information of the similar requirement.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: determining coverage of the test case by a mind map; regenerating the test case corresponding to the test requirement when the user triggers a regeneration operation.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: obtaining data for constructing the knowledge base from a data source; vectorizing the data to generate a knowledge vector and storing the knowledge vector in the knowledge base.

6. The method according to any one of claims 1 to 5, characterized in that, The knowledge base comprises at least one of a requirement knowledge base, a function knowledge base, a scenario knowledge base, a test point knowledge base, a problem list knowledge base, a document knowledge base, a test factor knowledge base or a test mode knowledge base.

7. The method according to any one of claims 1 to 6, characterized in that, The generating of the test case corresponding to the test requirement according to the at least one test point or the at least test-related factor comprises: providing the at least one test point or the at least one test-related factor to the user; determining at least one of a target test point or a target factor selected by the user; According to at least one of the target test point or the target factor, a test case corresponding to the test requirement is generated.

8. The method according to any one of claims 1 to 7, characterized in that, The test-related factor includes at least one of a test factor, a data factor, or an action factor.

9. A test system, characterized by The system includes: An interaction module configured to receive a test requirement of a user; A retrieval enhancement module configured to retrieve a knowledge base according to the test requirement, and obtain a test object related to the test requirement, the test object related to the test requirement including a test object of a similar requirement, the test object including at least one of a software and hardware code, a component, an interface, a document, or data, and the similar requirement including a requirement with a similarity to the test requirement of the user greater than a threshold value; A derivation module configured to derive, according to at least one of the test object related to the test requirement, a requirement document of the test requirement, or associated information including at least one of an associated function or a scenario, a use case generation auxiliary information through a language model, the use case generation auxiliary information including at least one of at least one test point or at least one test-related factor; A use case generation module configured to generate a test case corresponding to the test requirement according to the at least one test point or the at least one test-related factor.

10. The system of claim 9, wherein, The derivation module is further configured to: Derive, according to the test requirement or the associated information, a requirement type of the test requirement through the language model; Derive, according to the requirement type and the associated information, a test object of the test requirement through the language model; The retrieval enhancement module is specifically configured to: Enhance retrieval of the knowledge base according to at least one of the requirement type, a test object of the test requirement, a requirement document of the test requirement, or the associated information, and obtain a test object related to the test requirement.

11. The system of claim 9 or 10, wherein, The retrieval enhancement module is further configured to: Retrieve a knowledge base according to at least one of a requirement document of the test requirement or the associated information, and obtain at least one of a requirement document or the associated information of the similar requirement; The derivation module is specifically configured to: Derive, according to at least one of the test object related to the test requirement, the requirement document of the test requirement, or the associated information, the requirement document or the associated information of the similar requirement, a use case generation auxiliary information through a language model.

12. The system of any one of claims 9 to 11, wherein, The system further includes: An coverage estimation module configured to determine a coverage rate of the test case through a mind map; The use case generation module is specifically configured to: When the user triggers a regeneration operation, regenerate the test case corresponding to the test requirement.

13. The system of any one of claims 9 to 12, wherein, The system further includes: A knowledge base construction module configured to obtain data for constructing the knowledge base from a data source, vectorize the data to generate a knowledge vector, and store the knowledge vector in the knowledge base.

14. The system of any one of claims 9 to 13, wherein, The knowledge base includes at least one of a requirement knowledge base, a function knowledge base, a scenario knowledge base, a test point knowledge base, a problem list knowledge base, a document knowledge base, a test factor knowledge base, or a test mode knowledge base.

15. The system of any one of claims 9 to 14, wherein, The interaction module is further configured to: Provide the at least one test point or the at least one test-related factor to the user; determine at least one of a target test point or a target factor selected by the user; The use case generation module is specifically configured to: generate a test case corresponding to the test requirement according to at least one of the target test point or the target factor.

16. The system of any one of claims 9 to 15, wherein, The test-related factor includes at least one of a test factor, a data factor, or an action factor.

17. A cluster of computing devices, characterized in that, The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one memory stores computer readable instructions. The at least one processor executes the computer readable instructions to enable the computing device cluster to perform the test case generation method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer readable instructions are used to implement the test case generation method according to any one of claims 1 to 8.

19. A computer program product, characterised in that, The computer readable instructions are used to implement the test case generation method according to any one of claims 1 to 8.

Images