Test object test method and device and electronic equipment
By automatically generating test cases using large language models, the problem of time-consuming and labor-intensive manual test writing is solved, enabling efficient and accurate software testing, improving test coverage and depth, and reducing the illusion phenomenon.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-01
- Publication Date
- 2026-04-07
AI Technical Summary
Existing software testing methods rely on manually writing test cases, which is time-consuming and labor-intensive, makes it difficult to fully cover all test scenarios, and easily misses boundary conditions and abnormal paths, resulting in low testing efficiency, coverage, and depth.
The system introduces a large language model to automatically generate test cases. It obtains the original interface document, extracts parameter information, generates a parameter list and extraction report, combines sample test cases, generates test cases according to preset specifications, and corrects hallucination content through multi-stage prompt word templates and self-checking mechanisms.
Significantly reduces the time and cost of manually writing test cases, improves test efficiency, coverage and depth, ensures the accuracy and completeness of test cases, increases coverage by more than 30%, and reduces hallucination phenomena by 90%.
Smart Images
Figure CN121807690A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of artificial intelligence, and specifically relates to a test object testing method, device and electronic device. Background Technology
[0002] In practical applications, testing software products is a key step in ensuring their quality. Test cases are typically used to test software products. Specifically, a test case is a description of a testing task for a particular software product, reflecting the test plan, methods, techniques, and strategies. It is a set of test inputs, execution conditions, and expected results compiled for a specific goal to verify whether a particular software product requirement is met.
[0003] However, current software testing methods typically involve test engineers manually writing test cases, which is time-consuming, labor-intensive, and has high manpower costs. Furthermore, it is difficult to fully cover all test scenarios and is prone to missing boundary conditions and abnormal paths, resulting in low testing efficiency, coverage, and depth of software products. Summary of the Invention
[0004] The purpose of this application is to provide a test object testing method, apparatus, and electronic device to overcome or at least partially solve the above-mentioned problems.
[0005] To solve the above-mentioned technical problems, this application is implemented as follows: A test object testing method, the method comprising: Obtain the original interface documentation for the test object; The preset large language model is invoked to extract parameter information from the original interface document, and a parameter extraction report corresponding to the parameter information is generated. Generate a parameter list based on the parameter information and the parameter extraction report; Test cases are generated based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification. The test object is tested according to the test cases.
[0006] In the above embodiments, the system obtains the original interface document of the test object, calls a preset large language model to extract parameter information from the original interface document, generates a parameter extraction report corresponding to the parameter information, and generates a parameter list based on the parameter information and the parameter extraction report. Finally, test cases are generated based on the original interface document, the parameter list, and sample test cases to test the test object. The system of this application embodiment automatically generates test cases by introducing a large language model, which can significantly reduce the time and manpower cost of manually writing test cases, while maintaining or improving the testing efficiency, coverage, and testing depth of the test object.
[0007] In one embodiment of this application, after obtaining the original interface document of the test object, the method further includes: The original interface document is converted into a structured interface document in a structured format.
[0008] In the above embodiments, the system converts the original interface document of the software product into a structured interface document, such as converting the original interface document into a machine-readable standard format like JSON / YAML, which includes complete parameter names, data types, whether they are required, default values, and value ranges, so that the system and the large language model can accurately understand and process them.
[0009] In one embodiment of this application, generating a parameter list based on the parameter information and the parameter extraction report includes: Display the parameter information and the parameter extraction report; the parameter information includes at least the parameter name, data type, whether it is required, default value, and value range; In response to a calibration instruction submitted by a test engineer, the parameter information is calibrated according to the calibration instruction to generate a parameter list.
[0010] In the above embodiments, test engineers can view parameter information and parameter extraction reports through the human-computer interaction interface provided by the system to verify the accuracy and completeness of the parameter information. Furthermore, when it is determined that there are problems with the accuracy and completeness of the parameter information, a correction instruction can be submitted through the human-computer interaction interface. The system can then correct the parameter information according to the correction instruction, thereby forming the final parameter list used to generate test cases, ensuring the accuracy and completeness of the final generated test cases.
[0011] In one embodiment of this application, after generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, the method further includes: Determine the coverage of the test cases; When the coverage does not cover the preset test scenario, corresponding test cases are generated for the uncovered preset test scenario.
[0012] In the above embodiments, after generating test cases for the software product, if it is determined that the coverage of the test cases does not cover the preset test scenarios, the system will generate corresponding test cases for the uncovered preset test scenarios. This can determine the coverage of the test cases, provide a reference for the subsequent full generation of test cases, and improve the generation quality of test cases.
[0013] In one embodiment of this application, the step of generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, includes: Based on the original interface document, the parameter list, and the sample test cases, different types of test cases are generated in a preset order. The test cases include at least request example test cases, parameter combination test cases, single parameter test cases, enumeration value test cases, outlier value test cases, boundary value test cases, and null value test cases.
[0014] In the above embodiments, the system generates different types of test cases in a standardized preset order. This allows for the simultaneous and comprehensive verification of the software product using different types of test cases. Furthermore, the standardized test case generation process enables the planned and accurate generation of different types of test cases, thereby allowing for accurate testing of the software product using precise test cases.
[0015] In one embodiment of this application, generating different types of test cases according to the original interface document, the parameter list, and sample test cases in a preset order includes: Generate a multi-stage prompt word template based on the original interface document, the parameter list, and sample test cases; The preset large language model is invoked in stages according to a preset order to generate different types of test cases; wherein, the preset large language model uses the corresponding prompt word template to generate the corresponding type of test cases in each stage of test case generation.
[0016] In the above embodiments, a multi-stage prompt word template can be generated based on the original interface document, the parameter list, and sample test cases. Then, a preset large language model can be called in stages according to a preset order to generate different types of test cases. The preset large language model can use the corresponding prompt word template in the generation stage of each type of test case to generate the corresponding type of test cases. In this way, by designing multi-stage prompt word templates in stages, the large language model can be guided to generate high-quality test cases in accordance with the test specification requirements.
[0017] In one embodiment of this application, after generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, the method further includes: The preset large language model is invoked to check the test cases and the original interface documentation to determine whether the test cases contain illusory content; the illusory content includes at least non-existent parameters or values that violate constraints in the test cases.
[0018] In the above embodiments, a preset large language model is invoked to check the test cases and the original interface document to determine whether the test cases contain illusory content. This allows for the automatic triggering of a correction process when illusory content is detected, thereby ensuring the accuracy of the test cases ultimately used to test the software product.
[0019] A test object testing device, the device comprising: The original interface document retrieval module is used to retrieve the original interface document of the test object; The parameter information extraction module is used to call a preset large language model to extract parameter information from the original interface document and generate a parameter extraction report corresponding to the parameter information. The parameter list generation module is used to generate a parameter list based on the parameter information and the parameter extraction report. The test case generation module is used to generate test cases based on the original interface document, the parameter list and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification. The test object testing module is used to test the test object according to the test cases.
[0020] In one embodiment of this application, the apparatus further includes: a structured module, used for: The original interface document is converted into a structured interface document in a structured format.
[0021] In one embodiment of this application, the parameter list generation module is used for: Display the parameter information and the parameter extraction report; the parameter information includes at least the parameter name, data type, whether it is required, default value, and value range; In response to a calibration instruction submitted by a test engineer, the parameter information is calibrated according to the calibration instruction to generate a parameter list.
[0022] In one embodiment of this application, the apparatus further includes: a coverage checking module, used for: Determine the coverage of the test cases; When the coverage does not cover the preset test scenario, corresponding test cases are generated for the uncovered preset test scenario.
[0023] In one embodiment of this application, the test case generation module: Based on the original interface document, the parameter list, and the sample test cases, different types of test cases are generated in a preset order. The test cases include at least request example test cases, parameter combination test cases, single parameter test cases, enumeration value test cases, outlier value test cases, boundary value test cases, and null value test cases.
[0024] In one embodiment of this application, the test case generation module: Generate a multi-stage prompt word template based on the original interface document, the parameter list, and sample test cases; The preset large language model is invoked in stages according to a preset order to generate different types of test cases; wherein, the preset large language model uses the corresponding prompt word template to generate the corresponding type of test cases in each stage of test case generation.
[0025] In one embodiment of this application, the device further includes: a hallucination content checking module, used for: The preset large language model is invoked to check the test cases and the original interface documentation to determine whether the test cases contain illusory content; the illusory content includes at least non-existent parameters or values that violate constraints in the test cases.
[0026] An electronic device includes: a processor; and a memory for storing processor-executable instructions. The processor is configured to execute the instructions to implement the above-described test object testing method.
[0027] A computer-readable storage medium, when the instructions in the storage medium are executed by the processor of a mobile terminal, enables the mobile terminal to perform the aforementioned test object testing method.
[0028] The embodiments of this application have at least the following beneficial effects: In this embodiment, the system obtains the original interface document of the test object, calls a preset large language model to extract parameter information from the original interface document, generates a parameter extraction report corresponding to the parameter information, and generates a parameter list based on the parameter information and the parameter extraction report. Finally, test cases are generated based on the original interface document, parameter list, and sample test cases, and the test object is tested according to the test cases. The system in this embodiment, by introducing a large language model to automatically generate test cases, can significantly reduce the time and manpower costs of manually writing test cases, while maintaining or improving the testing efficiency, coverage, and depth of the test object. Attached Figure Description
[0029] Figure 1 This is a flowchart illustrating the steps of a test object testing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of an automatic test case generation process based on a large language model provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of a test device for a test object provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0030] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.
[0031] It should be noted that the embodiments of this application may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., with the user's explicit consent, with the user being properly notified, etc.).
[0032] Currently, the main method for testing software products is through manual coding, which has significant limitations. Specifically, it can be said that manually coding interface test cases is time-consuming and labor-intensive for test engineers, has high human resource costs, and is difficult to fully cover all test scenarios, easily missing boundary conditions and abnormal paths.
[0033] Currently, there is an emerging trend of applying large language models (large models) to testing. With the development of large language model technologies such as GPT, it has become possible to automatically generate test cases using AI (Artificial Intelligence), which is expected to significantly improve the testing efficiency of software products.
[0034] However, while large language models can automatically and quickly generate test cases, the current practice of directly applying them to test case generation often results in the "illusion problem," where the large language model generates parameters or values that do not conform to the actual interface definition. Currently, there is no mature technology in the industry that can systematically solve the illusion problem of large language models in test case generation, and existing methods struggle to guarantee the accuracy and quality of automatically generated test cases.
[0035] To address the aforementioned issues, this application provides an automated method for generating test cases that can effectively reduce the illusion of large language models and improve the quality of test cases for test objects. The system can maintain or improve the testing efficiency, coverage, and depth of test objects, thereby ensuring the accuracy and efficiency of testing test objects.
[0036] Reference Figure 1 The diagram illustrates a flowchart of a test object testing method provided in an embodiment of this application. The method specifically includes the following steps: Step 101: Obtain the original interface documentation of the test object.
[0037] In practice, the test object can be the entire software product or a specific functional module within it; this application's embodiments do not impose any limitations on this. For ease of explanation, the following description primarily uses the software product as the test object.
[0038] This application's embodiments can be applied to systems capable of generating test cases. Specifically, the system can simultaneously receive the original interface document of a software product. The original interface document may include an unstructured interface document describing the interaction specifications between components of the software product or between the system and external services, including interface addresses, request / response formats, parameter types, data structures, and error codes. Furthermore, the original interface document may also include a requirements document, which describes the functions, business rules, user goals, and performance requirements that the software product needs to implement.
[0039] Step 102: Invoke the preset large language model to extract parameter information from the original interface document and generate a parameter extraction report corresponding to the parameter information.
[0040] Large Language Models (LLMs), often simply referred to as large models, are deep learning models trained on large amounts of text data. Once trained, these models can generate natural language text or understand the meaning of spoken text, thus achieving semantic understanding. The system can include a model adaptation layer, an intermediate layer that allows the system to connect to different large language models, enabling flexible switching and invocation of various AI-powered large language models.
[0041] In this embodiment, a large language model trained on relevant training data for test cases is pre-deployed in the system. After obtaining the original interface document, the system calls the large language model API (Application Programming Interface) to extract all parameter information from the original interface document. The parameter information includes at least standardized parameters such as parameter name, data type, whether it is required, default value, and value range. Simultaneously, this embodiment can also generate a parameter extraction report corresponding to the parameter information. Specifically, the parameter extraction report lists each parameter extracted from the original interface document and its corresponding description in a way that is easy for testers to understand. The description may include the meaning, location, sensitivity, and importance of the parameter information. The description can be adjusted according to the tester's settings; for example, it may only describe whether the parameter information is sensitive. This embodiment does not impose any restrictions on this.
[0042] Step 103: Generate a parameter list based on the parameter information and the parameter extraction report.
[0043] In this embodiment of the application, after the system obtains parameter information and parameter extraction report from the original document through the large language model, it can generate a parameter list based on the parameter information and parameter extraction report. The parameter list consists of the parameters finally confirmed in the parameter information that are used to generate test cases.
[0044] Step 104: Generate test cases based on the original interface document, the parameter list, and the sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification.
[0045] Step 105: Test the test object according to the test cases.
[0046] In this embodiment, the system automatically generates several high-quality test cases using a rule-based approach, based on the original interface documentation, parameter list, a small number of pre-generated high-quality sample test cases, and preset test specifications. The software product can then be tested using these test cases. The sample test cases, generated using a rule-based approach based on the final parameter list and test specifications, cover various testing scenarios, including normal scenarios, boundary value testing, and outlier testing, serving as reference samples for generating more test cases from the large language model.
[0047] In the above embodiments, the system obtains the original interface document of the test object, calls a preset large language model to extract parameter information from the original interface document, generates a parameter extraction report corresponding to the parameter information, and generates a parameter list based on the parameter information and the parameter extraction report. Finally, test cases are generated based on the original interface document, the parameter list, and sample test cases to test the test object. The system of this application embodiment automatically generates test cases by introducing a large language model, which can significantly reduce the time and manpower cost of manually writing test cases, while maintaining or improving the testing efficiency, coverage, and testing depth of the test object.
[0048] In one embodiment of this application, after obtaining the original interface document of the test object, the method further includes: The original interface document is converted into a structured interface document in a structured format.
[0049] In practice, the original interface document is usually an unstructured document. When the system obtains the original interface document of the software product, it uses a large language model to convert it into a structured interface document in a structured format. For example, the large language model can be used to convert it into a structured format such as JSON (JavaScript Object Notation) / YAML (YAML Ain't Markup Language) to ensure information integrity.
[0050] In the above embodiments, the system converts the original interface document of the software product into a structured interface document, such as converting the original interface document into a machine-readable standard format like JSON / YAML, which includes complete parameter names, data types, whether they are required, default values, and value ranges. This can eliminate the ambiguity of the original document, reduce test case errors caused by interface misunderstandings, provide accurate input for the large language model, and facilitate accurate understanding and processing by the system and the large language model.
[0051] In one embodiment of this application, generating a parameter list based on the parameter information and the parameter extraction report may include: Display the parameter information and the parameter extraction report; the parameter information includes at least the parameter name, data type, whether it is required, default value, and value range; In response to a calibration instruction submitted by a test engineer, the parameter information is calibrated according to the calibration instruction to generate a parameter list.
[0052] In this embodiment, the system can display parameter information and parameter extraction reports through a human-computer interaction interface. The parameter information includes at least the parameter name, data type, whether it is required, default value, and value range. Test engineers can verify the accuracy and completeness of the parameter information by viewing the human-computer interaction interface. If the test engineer confirms that there are problems with these parameter information, he can submit a correction instruction through the human-computer interaction interface. After receiving the correction instruction, the system will correct the parameter information according to the correction instruction, thereby forming the final parameter list for generating test cases and storing it in the system database.
[0053] Furthermore, this application embodiment can also establish a traceable parameter verification mechanism. The generated parameter information, parameter extraction report, and the test engineer's correction of the parameter information will all be generated as logs and recorded, which facilitates subsequent traceability and ensures the instructions of the generated test cases.
[0054] In the above embodiments, test engineers can view parameter information and parameter extraction reports through the human-computer interaction interface provided by the system to verify the accuracy and completeness of the parameter information. Furthermore, when it is determined that there are problems with the accuracy and completeness of the parameter information, a correction instruction can be submitted through the human-computer interaction interface. The system can then correct the parameter information according to the correction instruction, thereby forming the final parameter list used to generate test cases, ensuring the accuracy and completeness of the final generated test cases.
[0055] In one embodiment of this application, after generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, the method may further include: Determine the coverage of the test cases; When the coverage does not cover the preset test scenario, corresponding test cases are generated for the uncovered preset test scenario.
[0056] In this embodiment, after generating test cases for the software product, the system needs to check and determine the coverage of the test cases. It needs to ensure that the generated test cases cover various preset test scenarios, such as normal scenarios, boundary value tests, and outlier tests, providing a reference for subsequent full-scale test case generation and improving the quality of generated test cases. Specifically, when the coverage does not cover the preset test scenarios, corresponding test cases are generated for the uncovered preset test scenarios. Of course, when the coverage completely covers the preset test scenarios, no operation is required, or a simple prompt through the human-computer interaction interface indicating complete test case coverage is sufficient.
[0057] In the above embodiments, after generating test cases for the software product, if it is determined that the coverage of the test cases does not cover the preset test scenarios, the system will generate corresponding test cases for the uncovered preset test scenarios. This can determine the coverage of the test cases, provide a reference for the subsequent full generation of test cases, and improve the generation quality of test cases.
[0058] In one embodiment of this application, the step of generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, may include: Based on the original interface document, the parameter list, and the sample test cases, different types of test cases are generated in a preset order. The test cases include at least request example test cases, parameter combination test cases, single parameter test cases, enumeration value test cases, outlier value test cases, boundary value test cases, and null value test cases.
[0059] In this embodiment of the application, the system can design test cases applicable to different types to ensure the systematic nature of the generation process according to the test specification requirements. Specifically, this embodiment of the application can automatically generate complete test cases of various types, such as request example test cases, parameter combination test cases, single parameter test cases, enumeration value test cases, outlier value test cases, boundary value test cases, and null value test cases, based on the original interface document, parameter list, and sample test cases in a preset order.
[0060] Specifically, the system breaks down the test case generation process into multiple consecutive steps according to the testing specifications (request example test cases → parameter combination test cases → single parameter / enumeration value / outlier value / boundary value / null value test cases). Each step includes a self-checking mechanism to identify and correct illusory content, ensuring that the generated test cases are consistent with the actual interface definition. Generating different types of test cases according to a preset order can improve test case coverage by more than 30%, including boundary cases and exceptional cases.
[0061] In the above embodiments, the system generates different types of test cases in a standardized preset order. This allows for the simultaneous and comprehensive verification of the software product using different types of test cases. Furthermore, the standardized test case generation process enables the planned and accurate generation of different types of test cases, thereby allowing for accurate testing of the software product using precise test cases.
[0062] In one embodiment of this application, the step of generating different types of test cases according to the original interface document, the parameter list, and sample test cases in a preset order may include: Generate a multi-stage prompt word template based on the original interface document, the parameter list, and sample test cases; The preset large language model is invoked in stages according to a preset order to generate different types of test cases; wherein, the preset large language model uses the corresponding prompt word template to generate the corresponding type of test cases in each stage of test case generation.
[0063] In its implementation, multi-stage prompt engineering is a prompt word design method optimized for large language models. It includes explicit constraint instructions, structured interface documentation (original interface documentation), parameter lists, and a small number of sample test cases. By designing multi-stage prompt word templates in stages, it guides the large language model to generate high-quality test cases according to testing specifications. This application's embodiment organically integrates the original interface documentation, parameter lists, and a small number of sample test cases to generate multi-stage prompt word templates. This significantly reduces the illusion phenomenon of large language models, improving test case accuracy by over 90%. Based on the multi-stage prompt word templates, the large model can understand and follow the testing specifications and constraints of specific interfaces, achieving controllability and predictability in test case generation.
[0064] In the above embodiments, a multi-stage prompt word template can be generated based on the original interface document, the parameter list, and sample test cases. Then, a preset large language model can be called in stages according to a preset order to generate different types of test cases. The preset large language model can use the corresponding prompt word template in the generation stage of each type of test case to generate the corresponding type of test cases. In this way, by designing multi-stage prompt word templates in stages, the large language model can be guided to generate high-quality test cases in accordance with the test specification requirements.
[0065] In one embodiment of this application, after generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, the method may further include: The preset large language model is invoked to check the test cases and the original interface documentation to determine whether the test cases contain illusory content; the illusory content includes at least non-existent parameters or values that violate constraints in the test cases.
[0066] In practical implementation, the "large language model illusion" phenomenon refers to inaccurate, unrealistic, or fabricated information generated by large language models when generating content. In test case generation scenarios, this manifests as generating parameters that do not exist in the original interface definition or values that violate interface constraints, resulting in invalid or poor-quality test cases.
[0067] To address the aforementioned issues, the system in this application embodiment can invoke a large language model to check test cases and original interface documentation to determine if test cases contain illusory content. Upon detecting illusory content, a correction process is automatically triggered. Specifically, after generating test cases, a second review can be performed using the large language model (inputting the original interface documentation and test cases) to self-check the generated test cases. The self-check content can include core data related to the interface definition, including parameter-related and request structure-related data. Parameter-related data includes: parameter name, parameter type, parameter value range, and required / optional attributes. Request structure-related data includes: request method (e.g., GET / POST).
[0068] In the above embodiments, a preset large language model is invoked to check the test cases and the original interface document to determine whether the test cases contain illusory content. This allows for the automatic triggering of a correction process when illusory content is detected, thereby ensuring the accuracy of the test cases ultimately used to test the software product.
[0069] To enable those skilled in the art to better understand the embodiments of this application, a specific example is used for illustration below. (Refer to...) Figure 2 This is a schematic diagram of an automatic test case generation process based on a large language model provided in this application embodiment, which specifically includes the following steps: Step 1: Interface Documentation Structuring: Receive the original interface documentation and convert it into structured JSON / YAML format.
[0070] Step 2: Parameter Information Extraction and Verification: Use the large language model to extract all parameter information and generate a parameter extraction report; perform human-computer interaction verification and correct the final parameter list.
[0071] Step 3: Sample Test Case Generation: Based on the parameter list and test specifications, generate a small number of high-quality sample test cases covering multiple scenarios.
[0072] Step 4: Multi-stage prompt engineering construction: Build prompt word templates with clear constraints, integrating interface documentation, parameter lists, and sample test cases.
[0073] Step 5: Step-by-step test case generation: Generate different types of test cases (request examples, parameter combinations, boundary values, etc.) according to the test specifications, and detect and correct illusory content through the large language model self-checking mechanism.
[0074] Specifically, the automatic test case generation process based on a large language model can include the following steps: S1: Interface document structuring: S1.1 Document Conversion: Receives the original interface document and uses a large model to convert it into structured JSON / YAML format, ensuring information integrity.
[0075] S2: Parameter Extraction and Validation: S2.1 Large Model Parameter Extraction: The large language model API is used to extract all parameter information from the structured document, including standardized parameters such as parameter name, data type, required / unrestricted status, default value, and value range. S2.2 Interactive Validation: A parameter extraction report is displayed through a human-computer interaction interface, allowing test engineers to verify and confirm the accuracy and completeness of the extraction. S2.3 Parameter List Correction: Based on the validation results, the parameter information is corrected, a final parameter list is generated, and stored in the system database.
[0076] S3: Sample Test Case Generation: S3.1 Rule-based Example Creation: Based on the final parameter list and preset test specifications, several high-quality test case examples are automatically generated using a rule-based approach. S3.2 Scenario Coverage Design: Ensures that the generated examples cover various test scenarios such as normal scenarios, boundary value tests, and outlier tests, providing a reference for subsequent full-scale generation.
[0077] S4: Multi-stage Hint Engineering Construction: S4.1 Constraint Hint Design: Design hint templates containing clear instructions and constraints, explicitly requiring key restrictions such as "only using parameters defined in the documentation." S4.2 Hint Content Integration: Organically integrate structured interface documentation, parameter lists, and few-sample examples into the hints to form a complete hint engineering system. S4.3 Phased Strategy Formulation: Based on testing specification requirements, design multi-stage hint strategies applicable to the generation of different types of test cases to ensure the systematic nature of the generation process.
[0078] S5: Step-by-step test case generation: S5.1 Classification test case generation: Generate different types of test cases step by step according to a preset order: S5.2 Illusion detection and correction: Require the large language model to self-check whether the content contains non-existent parameters or values that violate constraints during the generation process, and automatically trigger the correction process when illusion content is detected.
[0079] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of this application.
[0080] Reference Figure 3 The diagram shows a structural block diagram of a test object testing device provided in an embodiment of this application. The device may specifically include the following modules: The original interface document acquisition module 301 is used to acquire the original interface document of the test object; The parameter information extraction module 302 is used to call a preset large language model to extract parameter information from the original interface document and generate a parameter extraction report corresponding to the parameter information. The parameter list generation module 303 is used to generate a parameter list based on the parameter information and the parameter extraction report; The test case generation module 304 is used to generate test cases based on the original interface document, the parameter list and the sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification. The test object test module 305 is used to test the test object according to the test cases.
[0081] In one embodiment of this application, the apparatus further includes: a structured module, used for: The original interface document is converted into a structured interface document in a structured format.
[0082] In one embodiment of this application, the parameter list generation module 303 is used for: Display the parameter information and the parameter extraction report; the parameter information includes at least the parameter name, data type, whether it is required, default value, and value range; In response to a calibration instruction submitted by a test engineer, the parameter information is calibrated according to the calibration instruction to generate a parameter list.
[0083] In one embodiment of this application, the apparatus further includes: a coverage checking module, used for: Determine the coverage of the test cases; When the coverage does not cover the preset test scenario, corresponding test cases are generated for the uncovered preset test scenario.
[0084] In one embodiment of this application, the test case generation module 304: Based on the original interface document, the parameter list, and the sample test cases, different types of test cases are generated in a preset order. The test cases include at least request example test cases, parameter combination test cases, single parameter test cases, enumeration value test cases, outlier value test cases, boundary value test cases, and null value test cases.
[0085] In one embodiment of this application, the test case generation module 304: Generate a multi-stage prompt word template based on the original interface document, the parameter list, and sample test cases; The preset large language model is invoked in stages according to a preset order to generate different types of test cases; wherein, the preset large language model uses the corresponding prompt word template to generate the corresponding type of test cases in each stage of test case generation.
[0086] In one embodiment of this application, the device further includes: a hallucination content checking module, used for: The preset large language model is invoked to check the test cases and the original interface documentation to determine whether the test cases contain illusory content; the illusory content includes at least non-existent parameters or values that violate constraints in the test cases.
[0087] In this embodiment, the system obtains the original interface document of the test object, calls a preset large language model to extract parameter information from the original interface document, generates a parameter extraction report corresponding to the parameter information, and generates a parameter list based on the parameter information and the parameter extraction report. Finally, test cases are generated based on the original interface document, parameter list, and sample test cases to test the test object. The system in this embodiment automatically generates test cases by introducing a large language model, increasing the automation rate by 80%. This significantly reduces the time and manpower cost of manually writing test cases, while maintaining or improving the testing efficiency, coverage, and depth of the test object.
[0088] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment. This application also provides an electronic device, such as... Figure 4 As shown, it includes a processor 1001, a device interface 1002, a memory 1003, and a bus 1004; Memory 1003 is used to store computer programs; The processor 1001 executes the above steps when executing the program stored in the memory 1003.
[0089] The bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0090] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0091] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0092] This application also provides a storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the test object testing method of the foregoing embodiments.
[0093] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.
[0094] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. The structure required to construct such a device is obvious from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages is for the purpose of disclosing the best mode of implementation of this application.
[0095] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0096] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.
[0097] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.
[0098] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sequencing device according to this application. This application can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0099] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
[0100] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0101] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.
[0102] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0103] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.
Claims
1. A testing method for a test object, characterized in that, The method includes: Obtain the original interface documentation for the test object; The preset large language model is invoked to extract parameter information from the original interface document, and a parameter extraction report corresponding to the parameter information is generated. Generate a parameter list based on the parameter information and the parameter extraction report; Test cases are generated based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification. The test object is tested according to the test cases.
2. The method according to claim 1, characterized in that, After obtaining the original interface documentation of the test object, the method further includes: The original interface document is converted into a structured interface document in a structured format.
3. The method according to claim 1, characterized in that, The step of generating a parameter list based on the parameter information and the parameter extraction report includes: Display the parameter information and the parameter extraction report; the parameter information includes at least the parameter name, data type, whether it is required, default value, and value range; In response to a calibration instruction submitted by a test engineer, the parameter information is calibrated according to the calibration instruction to generate a parameter list.
4. The method according to claim 1, characterized in that, After generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, the method further includes: Determine the coverage of the test cases; When the coverage does not cover the preset test scenario, corresponding test cases are generated for the uncovered preset test scenario.
5. The method according to claim 1, characterized in that, The step of generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, includes: Based on the original interface document, the parameter list, and the sample test cases, different types of test cases are generated in a preset order. The test cases include at least request example test cases, parameter combination test cases, single parameter test cases, enumeration value test cases, outlier value test cases, boundary value test cases, and null value test cases.
6. The method according to claim 5, characterized in that, The process of generating different types of test cases according to the original interface document, the parameter list, and sample test cases in a preset order includes: Generate a multi-stage prompt word template based on the original interface document, the parameter list, and sample test cases; The preset large language model is invoked in stages according to a preset order to generate different types of test cases; wherein, the preset large language model uses the corresponding prompt word template to generate the corresponding type of test cases in each stage of test case generation.
7. The method according to claim 1, characterized in that, After generating test cases based on the original interface document, the parameter list, and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification, the method further includes: The preset large language model is invoked to check the test cases and the original interface documentation to determine whether the test cases contain illusory content; the illusory content includes at least non-existent parameters or values that violate constraints in the test cases.
8. A testing device for a test object, characterized in that, The device includes: The original interface document retrieval module is used to retrieve the original interface document of the test object; The parameter information extraction module is used to call a preset large language model to extract parameter information from the original interface document and generate a parameter extraction report corresponding to the parameter information. The parameter list generation module is used to generate a parameter list based on the parameter information and the parameter extraction report. The test case generation module is used to generate test cases based on the original interface document, the parameter list and sample test cases, wherein the sample test cases are generated based on the original interface document and the parameter list according to a preset test specification. The test object testing module is used to test the test object according to the test cases.
9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the test object testing method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal is able to perform the test object test method as described in any one of claims 1 to 7.