AI model-based software test case intelligent generation system and method

Abstract

The application discloses an AI model-based software test case intelligent generation system and method, wherein a user submits a test requirement through a Dify platform; the system automatically retrieves relevant data from a knowledge base management component; a DeepSeek LLM arrangement component generates a test case specification according to the requirement and the reference data; a parameter extractor analyzes LLM output, extracts test data and API parameters; an HTTP request component executes API testing and records results; a Python script generates an Excel test report according to a preset template; and the system directly replies to the user through the Dify platform and provides an Excel download link. The scheme reduces the dependence on manual test case writing, improves the effectiveness of defect discovery of test cases, realizes standardized output of generated test case templates, and facilitates test team cooperation and management.

Description

Technical Field

[0001] This invention relates to the field of software testing and artificial intelligence, and in particular to a system and method for intelligent generation of software test cases based on AI models. Background Technology

[0002] In the software development lifecycle, test case design and writing are crucial steps in ensuring software quality. Traditional manual test case writing suffers from low efficiency, incomplete coverage, and reliance on experience. With varying software complexity, agile iterative project cycles, and increasing demands for test quality, there is a pressing need for intelligent test case generation methods to meet the demands of rapid iterative development and improve testing efficiency. Summary of the Invention

[0003] To address the aforementioned technical issues, this invention provides an intelligent software test case generation system and method based on an AI model. By intelligently generating test cases, it reduces reliance on manually written test cases, improves the effectiveness of test cases in discovering defects, and achieves standardized output of generated test case templates, facilitating collaboration and management within the testing team.

[0004] This invention is achieved using the following technical solution: The first aspect is an AI-based intelligent software test case generation system, including: Knowledge base management component: Provides contextual support for DeepSeek LLM orchestration component, and uses search results as reference material for Prompt to ensure that the test cases generated by LLM meet the requirements document and test standards; Parameter extractor: Provides usable test data for HTTP request components, extracts and validates API parameters and uses them directly to build requests; DeepSeek LLM orchestration component: provides the parameter extractor with the original data source, generates structured test case text as the basis for parameter extraction; configures a multi-turn dialogue mechanism to respond to the correction request when the parameter extractor fails to verify, and regenerates the corrected test case text; HTTP request component: Provides test result data for the Excel template generation component and synchronizes request and response details to the Excel template generation component; Excel template generation component: Generates the output report.

[0005] Specifically, the knowledge base management component, DeepSeek LLM orchestration component, parameter extractor, HTTP request component, and Excel template generation component implement flow control through the orchestration flow engine of the Dify platform. The engine triggers component execution according to the preset node order and supports conditional branching.

[0006] Specifically, the system is also configured with an exception handling mechanism. When any component encounters an exception, the engine will report the exception to the preceding component, triggering a retry or user prompt.

[0007] Specifically, the knowledge base management component includes: Based on the underlying architecture of the Dify platform knowledge base, a document chunking storage mechanism is adopted to split the uploaded requirement documents and API specifications into fragments of preset size characters according to semantic logic; Embedded vector generation relies on the text encoding capabilities of the DeepSeek LLM orchestration component, which uses the model's Transformer encoder to transform text fragments into dense vectors of a preset dimension to capture semantic features. The index is built using the FAISS vector database, and an inverted index is built based on the ANN approximate nearest neighbor algorithm. It supports millisecond-level similarity retrieval, and the sorting is based on the cosine similarity algorithm to calculate the matching degree between vectors.

[0008] Specifically, the DeepSeekLLM orchestration component includes: Prompt template: It adopts a structured instruction design, including requirements parsing guidance, test case format constraints and output field definitions, to ensure the standardization of model generation results; Multi-turn dialogue mechanism: Based on conversation state management technology, it enables the hierarchical decomposition and progressive generation of complex requirements by storing historical dialogue contexts containing user supplementary information and model follow-up questions and responses; Function calls: Based on the Prompt project and tool binding, the function call format is preset in the Prompt (the model outputs structured data that conforms to JSON Schema, which can be directly connected to subsequent components).

[0009] Specifically, the parameter extractor includes: Regular expressions: Preset dedicated matching rules for API parameters and test data formats to quickly extract parameters with fixed formats; NER technology: Employs a named entity recognition fine-tuning model based on DeepSeek LLM orchestration components, specifically trained for entities in the software testing domain; Data validation: Based on syntax parsing and constraint rule engine, it first checks the data type matching degree, then verifies the parameter constraints, and returns a correction prompt if the conditions are not met.

[0010] Specifically, the HTTP request component includes: Requests are generated using dynamic parameter binding, which automatically maps the extracted API parameters to the URL path, request header, or request body according to the request method. The system parses responses based on JSONPath / XPath syntax, pre-sets key information extraction rules, and automatically determines whether the request was successful and whether the test results meet expectations. Configure timeout retry and exception capture mechanisms, set automatic retry counts for network fluctuations and interface timeouts, capture 4xx / 5xx status codes and record exception details.

[0011] Specifically, the Excel template generation component includes: Custom templates: The system uses a template file pre-storage and parameter replacement mechanism. Users upload an Excel template with preset headers, and the system fills in data in batches according to field matching. Visualization module: Visualizes test case marking based on conditional formatting rendering, and automatically generates test case pass rate statistics charts that are embedded in Excel files.

[0012] On the other hand, the AI-based intelligent generation method for software test cases includes the following steps: Step S1: Users submit test requirements through the Dify platform; Step S2: The system automatically retrieves relevant information from the knowledge base management component; Step S3: The system's DeepSeek LLM orchestration component generates test case specifications based on requirements and reference materials; Step S4: The parameter extractor parses the LLM output and extracts test data and API parameters; if parameter verification fails, the system parameter extractor sends a correction request to the DeepSeek LLM orchestration component, triggering step S3 to regenerate the corrected test case specification; Step S5: The HTTP request component performs API tests and records the results; Step S6: The Python script generates an Excel test report according to a preset template; Step S7: The system directly replies to the user through the Dify platform and provides an Excel download link.

[0013] The beneficial effects of this invention are as follows: (1) Improve efficiency and reduce costs: Traditional manual test case writing requires repeated reading of requirements and sorting out of logic, which is time-consuming and labor-intensive. The Dify orchestration process of this solution uses LLM automation generation + knowledge retrieval assistance to quickly transform natural language requirements into standardized test cases, which greatly reduces the cycle from requirements to test cases. For example, writing test cases for complex functional modules may take 1-2 days manually, but the Dify process can shorten it to hours, allowing the testing team to focus more on in-depth testing; (2) Social value: It releases productivity to support mass production. For projects with frequent iterations (such as agile development), it can automatically update use cases as requirements change, reducing manual maintenance costs. It accelerates iteration, improves production efficiency, and indirectly promotes the operation of the social economy; (3) Lower the technical threshold for test case writing, so that non-professional testers can also participate in test case generation. Test cases can be generated by describing requirements in natural language, breaking down the collaboration barriers between testing and business, and promoting closer teamwork. This solution achieves faster test case generation, shorter software iteration cycles, and more comprehensive and standardized test cases for high-quality products; thereby reducing the influx of software defects into the market and lowering the social costs associated with software failures. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0015] Figure 1 This is a system architecture diagram of an AI-based intelligent generation system for software test cases in an embodiment of the present invention. Figure 2 This is a schematic diagram of the intelligent generation method for software test cases based on AI models in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0017] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0018] The following is in conjunction with the appendix Figure 1-2 The following describes some embodiments of the present invention in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0019] This invention proposes an intelligent software test case generation system and method based on AI models. In a preferred embodiment, the system architecture is as follows: Figure 1 As shown, the core components and implementation include: (1) Knowledge base management component: Based on the underlying architecture of the Dify platform's knowledge base, a document chunking storage mechanism is adopted to split uploaded requirement documents, API specifications, etc., into fragments of 500-1000 characters according to semantic logic, thereby improving retrieval efficiency.

[0020] Embedded vector generation relies on the text encoding capabilities of DeepSeek LLM, which uses the model's Transformer encoder to transform text fragments into 768-dimensional dense vectors, capturing semantic features rather than literal matching.

[0021] The index is built using the FAISS vector database, and an inverted index is constructed based on the Approximate Nearest Neighbor (ANN) algorithm. It supports millisecond-level similarity retrieval, and the sorting is based on the cosine similarity algorithm to calculate the matching degree between vectors. (2) DeepSeek LLM orchestration component: The Prompt template adopts a structured instruction design, which includes three parts: requirement analysis guidance, test case format constraints, and output field definitions, to ensure the standardization of model generation results.

[0022] The multi-turn dialogue mechanism is based on conversation state management technology. By storing historical dialogue context (including user needs supplements and model follow-up questions and responses), it enables the hierarchical decomposition and progressive generation of complex needs.

[0023] Function call capability relies on Prompt project + tool binding technology. The function call format (such as parameter name, data type, and required fields) is preset in Prompt. The model outputs structured data that conforms to JSON Schema, which can be directly connected to subsequent components.

[0024] (3) Parameter extractor: Regular expressions have preset matching rules for API parameters and test data formats (such as URLs, phone numbers, dates, and numerical ranges) to enable the rapid extraction of parameters with fixed formats.

[0025] NER technology employs a fine-tuned model for named entity recognition based on DeepSeek LLM, specifically trained for entities in the software testing domain (such as interface names, request methods, response codes, and parameter types), achieving an accuracy rate of ≥95%.

[0026] Data validation is based on a syntax parsing and constraint rule engine. It first checks the matching degree of data types (such as strings, integers, and booleans), and then verifies parameter constraints (such as length, value range, and required fields). If the constraints are not met, a correction prompt is returned.

[0027] (4) HTTP request component: The request generation uses dynamic parameter binding technology, which automatically maps the extracted API parameters to the URL path, request header (such as Content-Type, Authorization) or request body (JSON / Form Data format) according to the request method (GET / POST / PUT / DELETE).

[0028] The response parsing is based on JSON Path / XPath syntax, with preset rules for extracting key information (such as response status codes, core returned fields, and error messages), and automatically determines whether the request was successful and whether the test results meet expectations.

[0029] It supports timeout retry and exception capture mechanisms, automatically retrying twice for network fluctuations and interface timeouts (default 30 seconds), capturing 4xx / 5xx status codes and recording exception details.

[0030] (5) Excel template generation component: Based on the openpyxl library, this low-level file operation API supports Excel workbook creation, worksheet addition, cell formatting (font, border, alignment), and formula embedding (such as test case execution result statistics).

[0031] The custom template uses a template file pre-storage + parameter replacement mechanism. Users can upload an Excel template with preset headers (such as test case ID, test scenario, input parameters, expected results, and actual results), and the system will fill in the data in batches according to the field matching.

[0032] Visualization relies on conditional formatting rendering technology, such as marking failed test cases in red and successful test cases in green, and automatically generating a test case pass rate statistical chart (pie chart) to be embedded in an Excel file.

[0033] The specific interaction flow of this system is as follows: user requirement submission → knowledge base management component retrieval → DeepSeekLLM orchestration component generates test cases → parameter extractor parses parameters → HTTP request component executes tests → Excel template generation component outputs reports.

[0034] The knowledge base management component provides contextual support for the DeepSeek LLM orchestration component, and the search results serve as reference material for the Prompt, ensuring that the test cases generated by the LLM conform to the requirements document and test standards.

[0035] The DeepSeek LLM orchestration component provides the raw data source for the parameter extractor. The structured use case text it generates forms the basis for parameter extraction, and the multi-turn dialogue mechanism can respond to the parameter extractor's correction requests (such as supplementing missing parameters).

[0036] The parameter extractor provides usable test data for the HTTP request component. The extracted and validated API parameters are directly used to build the request, avoiding test failures due to incorrect parameters.

[0037] The HTTP request component provides test result data to the Excel template generation component, synchronizing request response details (such as response time, returned data, and test conclusions) to Excel, thus completing the "test case generation-execution-recording" closed loop.

[0038] Each component uses the orchestration engine of the Dify platform to achieve flow control. The engine triggers the execution of components according to the preset node order and supports conditional branches (such as returning to the user to supplement requirements if parameter validation fails).

[0039] Collaborative exception handling: If any component encounters an exception (such as no results found in the knowledge base search, failure to generate an LLM, or an HTTP request timeout), the engine will report the exception to the preceding component, triggering a retry or user prompt, thus ensuring the stability of the process.

[0040] This embodiment also provides a specific module referenced by the Python script: import pandas as pd from flask import Flask, request, jsonify, send_from_directory from io import BytesIO import os import uuid import datetime from datetime from flask_cors import CORS This invention also proposes a method for intelligent generation of software test cases based on AI models, such as... Figure 2 As shown, it includes: Step S1: Users submit test requirements through the Dify platform; Step S2: The system automatically retrieves relevant information from the knowledge base management component; Step S3: The system's DeepSeek LLM orchestration component generates test case specifications based on requirements and reference materials; Step S4: The parameter extractor parses the LLM output and extracts test data and API parameters; if parameter verification fails, the system parameter extractor sends a correction request to the DeepSeek LLM orchestration component, triggering step S3 to regenerate the corrected test case specification; Step S5: The HTTP request component performs API tests and records the results; Step S6: The Python script generates an Excel test report according to a preset template; Step S7: The system directly replies to the user through the Dify platform and provides an Excel download link.

[0041] In one specific use case, the user logs into the open-source Dify platform, creates an application within Dify, adds usable models to the application, adds LLM, knowledge retrieval, and parameter extractor nodes to the orchestration flow, creates a knowledge base, adds requirements and other content to the knowledge base, adds this knowledge base to the application-knowledge retrieval section, and configures the address and parameters in the HTTP request; POSTs test case data to the backend API storage, calls a Python script to generate a service, converts the test cases into Excel, and finally returns the access link to the user; the orchestration is "driven" by user requirements, first finding the knowledge base, then having the LLM generate content, then processing the format / interfacing with external systems, and finally giving the results to the user or storing them in the system.

[0042] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0043] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A software test case intelligent generation system based on AI models, characterized in that, include: Knowledge base management component: Provides contextual support for DeepSeek LLM orchestration component, and uses search results as reference material for Prompt to ensure that the test cases generated by LLM meet the requirements document and test standards; Parameter extractor: Provides usable test data for HTTP request components, extracts and validates API parameters and uses them directly to build requests; DeepSeek LLM orchestration component: provides the raw data source for the parameter extractor and generates structured use case text as the basis for parameter extraction; Configure a multi-turn dialogue mechanism to respond to correction requests when the parameter extractor fails to verify, and regenerate the corrected test case text. HTTP request component: Provides test result data for the Excel template generation component and synchronizes request and response details to the Excel template generation component; Excel template generation component: Generates the output report.

2. The intelligent software test case generation system based on AI models as described in claim 1, characterized in that, The knowledge base management component, DeepSeek LLM orchestration component, parameter extractor, HTTP request component, and Excel template generation component are controlled through the orchestration flow engine of the Dify platform. The engine triggers the execution of components according to the preset node order and supports conditional branching.

3. The intelligent software test case generation system based on an AI model as described in claim 2, characterized in that, The system is also configured with an exception handling mechanism. When any component encounters an exception, the engine will report the exception to the preceding component, triggering a retry or user prompt.

4. The intelligent software test case generation system based on an AI model as described in claim 1, characterized in that, The knowledge base management component specifically includes: Based on the underlying architecture of the Dify platform knowledge base, a document chunking storage mechanism is adopted to split the uploaded requirement documents and API specifications into fragments of preset size characters according to semantic logic; Embedded vector generation relies on the text encoding capabilities of the DeepSeek LLM orchestration component, which uses the model's Transformer encoder to transform text fragments into dense vectors of a preset dimension to capture semantic features. The index is built using the FAISS vector database, and an inverted index is built based on the ANN approximate nearest neighbor algorithm. It supports millisecond-level similarity retrieval, and the sorting is based on the cosine similarity algorithm to calculate the matching degree between vectors.

5. The intelligent software test case generation system based on an AI model as described in claim 4, characterized in that, The DeepSeekLLM orchestration component specifically includes: Prompt template: It adopts a structured instruction design, including requirements parsing guidance, test case format constraints and output field definitions, to ensure the standardization of model generation results; Multi-turn dialogue mechanism: Based on conversation state management technology, it enables the hierarchical decomposition and progressive generation of complex requirements by storing historical dialogue contexts containing user supplementary information and model follow-up questions and responses; Function calls: Based on the Prompt project and tool binding, the function call format is preset in the Prompt (the model outputs structured data that conforms to JSON Schema, which can be directly connected to subsequent components).

6. The intelligent software test case generation system based on an AI model as described in claim 5, characterized in that, The parameter extractor specifically includes: Regular expressions: Preset dedicated matching rules for API parameters and test data formats to quickly extract parameters with fixed formats; NER technology: Employs a named entity recognition fine-tuning model based on DeepSeek LLM orchestration components, specifically trained for entities in the software testing domain; Data validation: Based on syntax parsing and constraint rule engine, it first checks the data type matching degree, then verifies the parameter constraints, and returns a correction prompt if the conditions are not met.

7. The AI-based intelligent software test case generation system as described in claim 6, characterized in that, The HTTP request component specifically includes: Requests are generated using dynamic parameter binding, which automatically maps the extracted API parameters to the URL path, request header, or request body according to the request method. The system parses responses based on JSONPath / XPath syntax, pre-sets key information extraction rules, and automatically determines whether the request was successful and whether the test results meet expectations. Configure timeout retry and exception capture mechanisms, set automatic retry counts for network fluctuations and interface timeouts, capture 4xx / 5xx status codes and record exception details.

8. The intelligent software test case generation system based on an AI model as described in claim 7, characterized in that, The Excel template generation component specifically includes: Custom templates: The system uses a template file pre-storage and parameter replacement mechanism. Users upload an Excel template with a preset header, and the system fills in data in batches according to field matching. Visualization module: Visualizes test case marking based on conditional formatting rendering, and automatically generates test case pass rate statistics charts that are embedded in Excel files.

9. A method for intelligently generating software test cases based on an AI model, implemented based on the intelligent software test case generation system based on an AI model as described in any one of claims 1 to 8, characterized in that, Includes the following steps: Step S1: Users submit test requirements through the Dify platform; Step S2: The system automatically retrieves relevant information from the knowledge base management component; Step S3: The system's DeepSeek LLM orchestration component generates test case specifications based on requirements and reference materials; Step S4: The parameter extractor parses the LLM output and extracts test data and API parameters; If parameter verification fails, the system parameter extractor sends a correction request to the DeepSeek LLM orchestration component, triggering step S3 to regenerate the corrected test case specification. Step S5: The HTTP request component performs API tests and records the results; Step S6: The Python script generates an Excel test report according to a preset template; Step S7: The system directly replies to the user through the Dify platform and provides an Excel download link.

Images