Rag and mcp-based structured test case generation method and system

Abstract

The application discloses a RAG and MCP-based structured test case generation method and system, and relates to the technical field of tool calling.The method comprises the following steps: constructing a RAG knowledge base, carrying out blocking, vectorization and storage on test-related documents, and storing a test parameter set.Making use of a test task parser, a user system-level test task is decomposed into an atomic task package.A RAG retrieval interface is called, the atomic task package text is converted into a vector query condition, relevant fragments are matched in the vector database, and the original text is reserved.A large language model extracts parameters from the test parameter set and generates structured test cases according to the matched fragments, the original text and generation requirements, calls an interface of a test case generator through an MCP protocol, and stores the structured test cases in a test case storage module for calling and execution by an automatic test framework.The method solves the problem that existing test case design mainly relies on manual experience and consumes a large amount of time and labor cost.

Description

Technical Field

[0001] This application relates to the field of tool invocation technology, and in particular to a method and system for generating structured test cases based on RAG and MCP. Background Technology

[0002] As modern software systems grow larger and more complex, the number of test cases that need to be designed for system testing also increases significantly.

[0003] Existing test case design mainly relies on human experience, requiring repeated reading of test outlines, requirements documents, design documents, interface specifications of the object under test, and other documents.

[0004] However, manually filling in or writing structured test cases according to fixed templates is not only time-consuming and labor-intensive, but also prone to incomplete test coverage or redundant test cases due to human error or misunderstanding. This inefficiency and high cost become even more pronounced when dealing with large-scale, highly complex software systems. Test engineers often spend a significant amount of time on repetitive document reading and test case writing, while the time actually devoted to test execution and problem localization is relatively limited, impacting the overall effectiveness of software testing and the software delivery cycle. Summary of the Invention

[0005] In this embodiment of the application, a structured test case generation method based on RAG and MCP is provided, which solves the problem that existing test case design mainly relies on human experience, which consumes a lot of time and manpower.

[0006] Firstly, embodiments of this application provide a structured test case generation method based on RAG and MCP. This method includes: constructing a RAG knowledge base; segmenting and vectorizing test requirement documents, design documents, test outlines, and test procedures, storing them in a vector database; preparing a test parameter set and storing it in the vector database; recursively decomposing user system-level test tasks into atomic function-level test tasks using a test task parser, generating atomic task packages containing original task description text and test case generation requirements; and converting the original task description text in the atomic task packages into... by calling the RAG retrieval interface in the RAG knowledge base. For vector query conditions, semantic similarity matching is performed in the vector database to obtain matching text fragments related to the original task description text, while retaining the original task description text. The RAG retrieval interface is used to enable interaction between the test task parser and the RAG knowledge base. Based on the matching text fragments, the original task description text, and test case generation requirements, the large language model calls the test case generator interface via the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure. These structured test cases are stored through the test case storage module interface for execution by the automated testing framework.

[0007] In one possible implementation, the construction of the RAG knowledge base involves segmenting and vectorizing the test requirement documents, design documents, test outlines, and test procedures, storing them in a vector database, and preparing a test parameter set, which is also stored in the vector database. This includes: cleaning and formatting the user-uploaded test requirement documents, design documents, test outlines, and test procedures using a document processing module, removing special symbols, garbled characters, and duplicate paragraphs, retaining valid information, and converting them to a unified format; splitting documents whose content exceeds the context length of the large language model into text fragments using a text segmentation module, setting the overlap of the fragment contexts, and obtaining segmented knowledge fragments; converting the segmented knowledge fragments into high-dimensional vectors using an embedding model using a vectorization processing module, obtaining vectorized text fragments; and storing the vectorized text fragments in the vector database and creating an index using a vector storage module.

[0008] In one possible implementation, the original task description text in the atomic task package is converted into vector query conditions in the RAG knowledge base by calling the RAG retrieval interface. Semantic similarity matching is then performed in the vector database to obtain matching text fragments related to the original task description text, while retaining the original task description text, including: the original task description text is presented in the form of natural language text, including the test target, test scope, and expected results; the retrieval module performs semantic similarity matching on the vector query conditions in the vector database to obtain the similarity between the vector query conditions and the vectorized text fragments in the vector database; the retrieval module filters out the vectorized text fragments with high similarity according to a pre-set similarity threshold and uses them as matching text fragments related to the original task description text.

[0009] In one possible implementation, the large language model, based on the matching text fragment, the original task description text, and the test case generation requirements, calls the interface of the test case generator via the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure. This includes: the large language model receiving the matching text fragment returned by the retrieval module; converting the original task description text and information from tools registered in the MCP adapter into vectors respectively; obtaining the similarity between two vectors; matching based on the obtained similarity; filtering tools with similarity higher than a threshold from the toolchain list provided in the MCP adapter; determining the interface of the test case generator; and calling the interface of the test case generator via the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure.

[0010] In one possible implementation, the step of filtering tools with similarity higher than a threshold from the toolchain list provided in the MCP adapter and determining the interface of the test case generator includes: if there is a tool with similarity higher than the threshold, determining its corresponding interface and using that interface as the interface of the test case generator; if there are multiple tools with similarity higher than the threshold, selecting the tool with the highest similarity, determining its corresponding interface, and using that interface as the interface of the test case generator.

[0011] In one possible implementation, storing structured test cases through the interface of the test case storage module includes: providing an interface with functions for adding, modifying, deleting, and querying test case storage modules through an interface adaptation module; the interface of the test case storage module is used to enable interaction between the test case storage module and external modules; when storing structured test cases, the external module initiates a storage request through the interface of the test case storage module; the interface adaptation module encapsulates the internal processing procedures and business logic, performs preliminary processing on the storage request, and passes the processed storage request to the data read / write module; the data read / write module executes the data writing task according to the preset conditions in the storage request passed by the interface adaptation module.

[0012] In one possible implementation, the automated testing framework executes structured test cases, specifically including: using the test case loading module to call the query function of the test case storage module's interface to read structured test cases; using the test case parsing module to parse the read structured test cases; the test case execution engine driving the automated testing engine to execute step by step according to the test steps, action instructions, parameters, and judgment conditions in the parsed structured test cases to obtain test results; and using the test report generation module to generate a test report based on the test results.

[0013] Secondly, embodiments of this application provide a structured test case generation system based on RAG and MCP. This system includes a test task parser, a RAG knowledge base, an MCP adapter, a test case generator, an automated testing framework, and a test case storage module. The test task parser is configured to recursively decompose user system-level test tasks into atomic functional-level test tasks, generating atomic task packages containing original task description text and test case generation requirements. The RAG knowledge base is configured to convert the original task description text in the atomic task packages into vector query conditions, perform semantic similarity matching in a vector database, obtain matching text fragments related to the original task description text, and retain the original task description text. The large language model, based on the matching text fragments, the original task description text, and the test case generation requirements, calls the interface of the test case generator through the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure. The automated testing framework is configured to call and execute the structured test cases. The test case storage module is configured to store the structured test cases.

[0014] Thirdly, embodiments of this application provide a structured test case generation server based on RAG and MCP, including a memory and a processor; the memory is used to store computer-executable instructions; the processor is used to execute the computer-executable instructions to implement the method described in the first aspect or any possible implementation of the first aspect.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing executable instructions, which, when executed by a computer, enable the method described in the first aspect or any possible implementation thereof.

[0016] One or more technical solutions provided in the embodiments of this application have at least the following technical effects:

[0017] This application provides a structured test case generation method based on RAG and MCP. This solution constructs a RAG knowledge base, segments and vectorizes various documents, and stores them in a vector database, achieving structured and digital management of document information. Testers no longer need to manually sift through large amounts of paper or electronic documents; they can quickly locate matching text fragments related to the original task description text through the RAG retrieval interface, greatly improving information retrieval efficiency. The test task parser can recursively decompose user system-level test tasks into atomic functional-level test tasks and generate atomic task packages containing the original task description text and test case generation requirements. This process eliminates the tedious manual decomposition of test tasks, avoids inaccurate task decomposition due to human misunderstanding, and improves task decomposition efficiency several times over. Based on matching text fragments, the original task description text, and test case generation requirements, the large language model calls the test case generator interface through the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure. The structured test cases are stored through the test case storage module interface for execution by the automated testing framework. Compared to the traditional method of manually filling in or writing test cases according to fixed templates, this application achieves automated generation of structured test cases, thus improving the generation speed. It solves the problem that existing test case design mainly relies on human experience, consuming significant time and manpower costs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a structured test case generation method based on RAG and MCP provided in this application embodiment.

[0020] Figure 2 This is a schematic diagram of the system architecture provided for an embodiment of this application.

[0021] Figure 3 This is a schematic diagram of a structured test case generation server based on RAG and MCP, provided for an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0023] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0024] This application provides a method for generating structured test cases based on RAG and MCP, such as Figure 1 As shown, the method includes steps S101 to S105. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order of a structured test case generation method based on RAG and MCP. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0025] Figure 2This is a schematic diagram of the system architecture provided in an embodiment of this application. Specifically, it includes a test task parser, a RAG knowledge base, an MCP adapter, a test case generator, an automated testing framework, and a test case storage module. The test task parser is configured to recursively decompose user system-level test tasks into atomic functional-level test tasks, generating atomic task packages containing original task description text and test case generation requirements. The RAG knowledge base is configured to convert the original task description text in the atomic task packages into vector query conditions, perform semantic similarity matching in the vector database, obtain matching text fragments related to the original task description text, and retain the original task description text. Based on the matching text fragments, the original task description text, and the test case generation requirements, the large language model calls the test case generator's interface through the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure. The automated testing framework is configured to call and execute the structured test cases. The test case storage module is configured to store structured test cases.

[0026] In this application, RAG stands for Retrieval-Augmented Generation, and MCP stands for Model-Context Protocol.

[0027] S101: Build a RAG knowledge base, divide and vectorize the test requirement document, design document, test outline and test process into blocks, store them in a vector database, prepare the test parameter set and store it in the vector database as well.

[0028] Build a RAG knowledge base, segment and vectorize the test requirement document, design document, test outline and test process, and store them in a vector database. Prepare the test parameter set and store it in the vector database as well, including the following content.

[0029] The document processing module cleans and formats the test requirement documents, design documents, test outlines, and test procedures uploaded by users, removing special symbols, garbled characters, and duplicate paragraphs, retaining valid information, and converting them into a unified format.

[0030] Specifically, a unified format mainly refers to the process of cleaning and transforming the text content of various documents into plain text (or lightweight markup language Markdown) data objects with a consistent structure. Its core characteristics are clean content, standardized structure, and accompanying source metadata. The purpose is to establish a high-quality and unambiguous starting point for subsequent intelligent text segmentation, vectorization, and retrieval.

[0031] Specifically, users can upload documents in various formats such as PDF and Word.

[0032] The text segmentation module splits documents whose content exceeds the context length of the large language model into text fragments, and sets the overlap of the fragment contexts to obtain the segmented knowledge fragments.

[0033] Specifically, overlap is the length of overlapping characters between adjacent text blocks when segmenting long text. By setting the overlap appropriately, the semantic fragmentation of the context can be effectively avoided, ensuring that key information across text blocks and complete sentences and phrases are preserved intact. Although there is no fixed value for overlap, it is positively correlated with the block size. The ratio of overlap O to block size S, O / S, is usually controlled between 10% and 25%. At the same time, it is necessary to flexibly adjust and optimize it based on the actual block size, text type, and business scenario to obtain the best segmentation effect and the segmented knowledge fragments.

[0034] The vectorization module converts the segmented knowledge fragments into high-dimensional vectors using the Embedding model, thus obtaining the vectorized text fragments.

[0035] Specifically, the Embedding model is a machine learning model that maps discrete data (such as words or characters in text, or pixels in an image) to vectors in a continuous vector space.

[0036] The vectorized text fragments are stored in the vector database and an index is created using the vector storage module.

[0037] S102: The test task parser recursively decomposes the user system-level test task into atomic functional-level test tasks, generating an atomic task package containing the original task description text and test case generation requirements.

[0038] Specifically, the test task parser includes a task decomposition module and a RAG interface management module.

[0039] The task decomposition module decomposes user system-level test tasks into atomic function-level test tasks based on the test task specification, and adds detailed original task description text and test case generation requirements to obtain atomic task packages.

[0040] The RAG interface management module calls the external retrieval interface of the RAG knowledge base and sends the original task description text and test case generation requirements generated by the task decomposition module to the retrieval module of the RAG knowledge base.

[0041] S103: By calling the RAG retrieval interface, the original task description text in the atomic task package is converted into vector query conditions in the RAG knowledge base. Semantic similarity matching is performed in the vector database to obtain matching text fragments related to the original task description text, while retaining the original task description text.

[0042] The RAG retrieval interface is used to enable interaction between the test task parser and the RAG knowledge base.

[0043] By calling the RAG retrieval interface, the original task description text in the atomic task package is converted into vector query conditions in the RAG knowledge base. Semantic similarity matching is performed in the vector database to obtain matching text fragments related to the original task description text, while retaining the original task description text, including the following content.

[0044] The original task description text is presented in the form of natural language text, including the test objective, test scope, and expected results.

[0045] The retrieval module performs semantic similarity matching on vector query conditions in the vector database to obtain the similarity between the vector query conditions and the vectorized text fragments in the vector database.

[0046] The retrieval module filters out vectorized text fragments with high similarity based on a pre-set similarity threshold and uses them as matching text fragments related to the original task description text.

[0047] Specifically, the retrieval module can employ a cosine similarity algorithm, calculating the cosine of the angle between two vectors to determine the similarity between the vector query conditions and the vectorized text fragments in the vector database. Cosine similarity measures the cosine of the angle between two vectors, with a similarity range from -1 to 1. -1 means the two vectors point in opposite directions, 1 indicates they point in exactly the same direction, 0 typically indicates they are independent, and values ​​in between represent intermediate similarity or dissimilarity. If the angle between two vectors is close to 0 degrees, the closer the cosine value is to 1, the more similar the two vectors are; if the angle is close to 90 degrees (cosine value close to 0), they are dissimilar; if the angle is close to 180 degrees (cosine value close to -1), they are highly dissimilar.

[0048] Specifically, a pre-set similarity threshold of 0.75 is used to filter all matching results, selecting vectorized text fragments with a similarity greater than 0.75. These vectorized text fragments are then considered the most similar and are considered the matching text fragments related to the original task description text. Throughout this process, the original task description text is preserved intact so that it can be sent along with the matching text fragments to the Large Language Model (LLM) for augmentation, further improving the accuracy and effectiveness of test case generation.

[0049] S104: The large language model, based on matching text fragments, original task description text, and test case generation requirements, calls the test case generator interface through the MCP protocol to extract parameters from the test parameter set and generate structured test cases that conform to the predefined data structure.

[0050] The large language model, based on matching text fragments, original task description text, and test case generation requirements, calls the test case generator interface through the MCP protocol to extract parameters from the test parameter set and generate structured test cases that conform to a predefined data structure, including the following:

[0051] The large language model receives the matching text fragments returned by the retrieval module and converts the original task description text and the information from the tools registered in the MCP adapter into vectors respectively.

[0052] Specifically, information in natural language form is converted into vector form that computers can process efficiently, laying the foundation for subsequent similarity matching.

[0053] The similarity between two vectors is obtained, and matching is performed based on the obtained similarity. Tools with similarity higher than the threshold are filtered from the toolchain list provided in the MCP adapter to determine the interface of the test case generator.

[0054] Specifically, the threshold value can be between 0.7 and 0.8.

[0055] Specifically, the similarity between two vectors can be obtained using the cosine similarity algorithm.

[0056] Filter tools with similarity higher than a threshold from the toolchain list provided in the MCP adapter to determine the interface of the test case generator, including the following:

[0057] If a tool with a similarity higher than the threshold exists, determine its corresponding interface and use that interface as the interface for the test case generator.

[0058] If multiple tools have similarity scores higher than the threshold, select the tool with the highest similarity score, determine its corresponding interface, and use that interface as the interface for the test case generator.

[0059] The test case generator's interface is called via the MCP protocol to extract parameters from the test parameter set and generate structured test cases that conform to a predefined data structure.

[0060] Specifically, the MCP adapter in this application includes an MCP toolchain registration module and an MCP toolchain query module.

[0061] The toolchain registration module has the function of configuring tool link interfaces that conform to MCP (a collection of tool interfaces in a toolchain). When a new tool needs to be integrated, staff can enter various information about the tool, such as its function description, interface type description, interface parameter requirements, return results, and invocation method, into this module. This configuration information is automatically loaded when the module starts, allowing the new tool to be successfully added to the toolchain list for subsequent use.

[0062] The toolchain query module primarily serves the large language model. It provides descriptions of the interfaces of all registered test case generators conforming to the MCP protocol, and also reports the current status of these interfaces, such as availability. The large language model can then use this information to query and invoke the appropriate test case generator interface to meet different task requirements.

[0063] Specifically, the test case generator in this application includes a format conversion module, a data correction module, a data storage module, and a data encapsulation module.

[0064] The test case generator is responsible for converting interface-input parameters into structured test case data and persistently storing it. It also allows users to modify and adjust the generated test cases. This test case generator provides an interface (i.e., the test case generator's interface) that conforms to the MCP protocol and is part of the MCP toolchain. It needs to be registered with the MCP adapter so that the Large Language Model (LLM) can call it autonomously.

[0065] The data encapsulation module is responsible for structured modeling of test case descriptions. It abstracts structures encompassing key information such as test case number, test steps, execution commands, parameters, and decision conditions. Its core function is to accurately encapsulate the parameters passed from the interface into test case structure objects, laying the foundation for subsequent processing. The format conversion module converts the test case structure objects generated by the data encapsulation module into standard format data, supporting conversion to JSON or XML. This conversion makes the test case data more universal and readable, facilitating interaction and processing across different systems and platforms. To meet users' personalized needs for test cases, the data correction module provides a user interface. Through this interface, users can flexibly modify and adjust the generated structured test case data, ensuring that the test cases accurately reflect actual testing requirements and improving the accuracy and effectiveness of testing. The data storage module calls the data addition interface provided by the test case storage module to persistently store the standard format test case data processed by the format conversion module. In this way, test case data is preserved long-term, facilitating subsequent querying, management, and reuse, providing strong support for the continuous development of testing work.

[0066] S105: Store structured test cases through the interface of the test case storage module for execution by the automated testing framework.

[0067] Structured test cases are stored through the interface of the test case storage module, including the following:

[0068] The interface adaptation module provides an interface for the test case storage module, which has functions for adding, modifying, deleting, and querying test cases. The interface of the test case storage module is used to enable the interaction between the test case storage module and external modules.

[0069] When storing structured test cases, external modules initiate storage requests through the test case storage module's interface. The interface adaptation module encapsulates the internal processing procedures and business logic, performs preliminary processing on the storage requests, and then passes the processed storage requests to the data read / write module.

[0070] The data read / write module executes data writing tasks based on the preset conditions in the storage request passed by the interface adaptation module.

[0071] The automated testing framework executes structured test cases, including the following:

[0072] The test case loading module calls the query function of the test case storage module's interface to read structured test cases.

[0073] The test case parsing module parses the read structured test cases.

[0074] The test case execution engine drives the automated test engine to execute step by step based on the test steps, action instructions, parameters, and judgment conditions in the parsed structured test cases, and obtains the test results.

[0075] The test report generation module generates a test report based on the test results.

[0076] Taking the hydraulic landing gear system test platform as an example, the specific construction process of the test platform is as follows.

[0077] 1. Construct a local LLM general-purpose large language model and a vectorized model. There are no restrictions on the construction method or specific model selection for the LLM and vectorized models; only that the LLM large language model supports the MCP protocol is required. This example uses Ollam as the private deployment platform for the model, employs the Qwen3-32B pre-trained model from Tongyi Qianwen for the LLM, and uses the Qwen3-Embedding-8B embedding model.

[0078] 2. Build a local vector database using Elasticsearch (an open-source distributed search and analytics engine).

[0079] 3. Building a RAG knowledge base: Existing open-source RAG knowledge base frameworks can be used for local deployment, such as Tencent WeKnora, Anything LLM, LangChain-Chatchat, Dify, RAGFlow, etc. This example uses Tencent WeKnora.

[0080] 4. The main configuration involves API access to LLM, vector models, and vector databases. It also includes setting parameters such as the type, size, and chunk size of uploaded documents, and configuring the API interface for providing vector retrieval services to complete the RAG knowledge base configuration.

[0081] 5. Configure the retrieval interface provided by the RAG knowledge base in the test task parser for subsequent calls.

[0082] 6. Expose an interface conforming to the MCP protocol in the test case generator, i.e., the interface of the test case generator.

[0083] 7. Register Interface Information: Register the MCP interface information provided by the test case generator in the MCP adapter, and share the list of MCP tool link interface information with LLM using the MCP toolchain query module.

[0084] 8. Using the document processing module of the RAG knowledge base, upload existing requirement documents, test outlines, design documents, historical test case documents, and interface documents of the tested object related to the automatic flight control system test, and complete the data cleaning and formatting of the documents.

[0085] 9. Using the text segmentation module of the RAG knowledge base, divide the cleaned and formatted test-related documents into text blocks. In this example, the size of each text block is set to 1024 tokens.

[0086] 10. Using the Qwen3-Embedding-8B vectorization model of the RAG knowledge base, the segmented text is vectorized, converting a piece of text (a segment) into a string of numbers (a high-dimensional vector). This vector can be regarded as the "semantic coordinates" of the text in mathematical space. Texts with similar semantics have vectors that are closer in position in space.

[0087] 11. Store the vectorized data in the Elasticsearch vector database and create an index for easy retrieval later.

[0088] 12. Create test tasks in the test task parser and decompose them into atomic-level test functions. For hydraulic landing gear system testing, the function can be decomposed into atomic functions such as landing gear retraction and extension, while adding test descriptions and test case generation requirements.

[0089] 13. Through the RAG interface management module, the test task description and test case generation requirements are used as parameters to call the RAG knowledge base retrieval interface.

[0090] 14. Through the retrieval module of the RAG knowledge base, perform vectorized search and comparison, and forward the test methods, steps, parameters and other information matched in the RAG knowledge base, along with the original test tasks and test case generation requirements, to the LLM for enhancement processing.

[0091] 15. LLM performs in-depth understanding and analysis of the original question and requirements (i.e., the original task description text), and optimizes and enhances them by combining the results returned by the knowledge base to obtain optimized answer content.

[0092] 16. After understanding the user's need to generate structured test cases, LLM analyzes the MCP toolchain list information to find the interface that can implement this function, namely the MCP protocol interface provided by the test case generator. LLM extracts parameters from the response content and organizes them into a form that conforms to the parameter requirements of the interface, and calls the interface to extract information such as test steps, test instructions, parameters, and judgment conditions.

[0093] Specifically, the parameters extracted from the answer need to be mapped to the format or values ​​defined in the test parameter set.

[0094] 17. The data encapsulation module of the test case generator processes the parameters passed from the interface, generates objects that meet the structure requirements, and formats the test steps, test instructions, parameters, and judgment conditions to generate structure objects.

[0095] 18. The format conversion module of the test case generator converts the generated structure object into JSON format data.

[0096] 19. Users can view the generated JSON-formatted structured test cases through the human-computer interaction interface provided by the data correction module of the test case generator, and optimize and adjust the JSON-formatted structured test case data.

[0097] 20. After the adjustment is completed, call the persistent storage interface through the data storage module of the test case generator to save the test cases in JSON format to the test case storage module.

[0098] 21. The automated testing framework can retrieve JSON-formatted test cases through the query interface provided by the test case storage module, and then load, parse, and execute them. The test case execution engine converts the instructions and parameters in the test steps into interface protocols such as 429 / 664 to achieve signal stimulation and acquisition, and obtains test results based on criteria, generating a test report.

[0099] Thus, the hydraulic landing gear system test platform built based on RAG and MCP technologies in this example can automatically generate structured test cases and drive the test framework to execute automated tests through structured test cases.

[0100] Some modules of the system described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via communication networks. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0101] The systems or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above systems are described by dividing them into various modules based on their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0102] The methods, systems, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. Memory controllers can also be implemented as part of the control logic of memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the systems included within it for implementing various functions can also be considered structures within the hardware component. Alternatively, a system used to implement various functions can be viewed as either a software module implementing the method or a structure within a hardware component.

[0103] like Figure 3 As shown in the figure, this application embodiment also provides a structured test case generation server based on RAG and MCP, including a memory 301 and a processor 302; the memory 301 is used to store computer-executable instructions; the processor 302 is used to execute computer-executable instructions to implement the structured test case generation method based on RAG and MCP described above in this application embodiment.

[0104] This application also provides a computer-readable storage medium storing executable instructions, which, when executed by a computer, enable the structured test case generation method based on RAG and MCP described above in this application.

[0105] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the embodiments of this application.

[0106] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations.

[0107] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for generating structured test cases based on RAG and MCP, characterized in that, include: Build a RAG knowledge base, divide and vectorize the test requirement document, design document, test outline and test process into blocks and store them in a vector database, prepare the test parameter set and store it in the vector database as well; The test task parser recursively decomposes user system-level test tasks into atomic function-level test tasks, generating atomic task packages containing original task description text and test case generation requirements. By calling the RAG retrieval interface, the original task description text in the atomic task package is converted into vector query conditions in the RAG knowledge base. Semantic similarity matching is performed in the vector database to obtain matching text fragments related to the original task description text, while retaining the original task description text. The RAG retrieval interface is used to realize the interaction between the test task parser and the RAG knowledge base. The large language model, based on matching text fragments, original task description text, and test case generation requirements, calls the test case generator interface through the MCP protocol to extract parameters from the test parameter set and generate structured test cases that conform to the predefined data structure. Structured test cases are stored through the interface of the test case storage module for execution by the automated testing framework. The construction of the RAG knowledge base involves segmenting and vectorizing the test requirement document, design document, test outline, and test process, storing them in a vector database, and preparing the test parameter set, which is also stored in the vector database. The document processing module cleans and formats the test requirement documents, design documents, test outlines and test procedures uploaded by users, removing special symbols, garbled characters and repeated paragraphs, retaining valid information and converting them into a uniform format. The text segmentation module splits documents whose content exceeds the context length of the large language model into text fragments, and sets the overlap of the fragment contexts to obtain the segmented knowledge fragments. The vectorization module converts the segmented knowledge fragments into high-dimensional vectors using the Embedding model, thus obtaining the vectorized text fragments. The vectorized text fragments are stored in the vector database and an index is created using the vector storage module. The process involves calling the RAG retrieval interface to convert the original task description text in the atomic task package into vector query conditions in the RAG knowledge base, performing semantic similarity matching in the vector database, obtaining matching text fragments related to the original task description text, and retaining the original task description text, including: The original task description text is presented in the form of natural language text, including the test objectives, test scope, and expected results; The retrieval module performs semantic similarity matching on the vector query conditions in the vector database to obtain the similarity between the vector query conditions and the vectorized text fragments in the vector database. The retrieval module filters out vectorized text fragments with high similarity based on a pre-set similarity threshold and uses them as matching text fragments related to the original task description text. The large language model, based on matching text fragments, original task description text, and test case generation requirements, calls the test case generator interface via the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure, including: The large language model receives the matching text fragments returned by the retrieval module and converts the original task description text and the information from the tools registered in the MCP adapter into vectors respectively. Obtain the similarity between two vectors, perform matching based on the obtained similarity, filter tools with similarity higher than the threshold from the toolchain list provided in the MCP adapter, and determine the interface of the test case generator; The test case generator's interface is called via the MCP protocol to extract parameters from the test parameter set and generate structured test cases that conform to a predefined data structure.

2. The structured test case generation method based on RAG and MCP according to claim 1, characterized in that, The process of filtering tools with similarity higher than a threshold from the toolchain list provided in the MCP adapter to determine the interface of the test case generator includes: If a tool with a similarity higher than the threshold exists, determine its corresponding interface and use that interface as the interface for the test case generator. If multiple tools have similarity scores higher than the threshold, select the tool with the highest similarity score, determine its corresponding interface, and use that interface as the interface for the test case generator.

3. The structured test case generation method based on RAG and MCP according to claim 1, characterized in that, The process of storing structured test cases through the interface of the test case storage module includes: The interface adaptation module provides an interface for the test case storage module with functions for adding, modifying, deleting, and querying test cases. The interface of the test case storage module is used to enable the interaction between the test case storage module and external modules. When storing structured test cases, external modules initiate storage requests through the test case storage module's interface. The interface adaptation module encapsulates the internal processing procedures and business logic, performs preliminary processing on the storage requests, and then passes the processed storage requests to the data read / write module. The data read / write module executes data writing tasks based on the preset conditions in the storage request passed by the interface adaptation module.

4. The structured test case generation method based on RAG and MCP according to claim 3, characterized in that, The automated testing framework invokes and executes structured test cases, specifically including: The test case loading module calls the query function of the test case storage module's interface to read structured test cases; The structured test cases are parsed using the test case parsing module. The test case execution engine drives the automated test engine to execute step by step based on the test steps, action instructions, parameters and judgment conditions in the parsed structured test cases to obtain test results; The test report generation module generates a test report based on the test results.

5. A structured test case generation system based on RAG and MCP, characterized in that, The system performs the method as described in any one of claims 1 to 4, including a test task parser, a RAG knowledge base, an MCP adapter, a test case generator, an automated testing framework, and a test case storage module; The test task parser is configured to recursively decompose user system-level test tasks into atomic functional-level test tasks, generating atomic task packages containing original task description text and test case generation requirements. The RAG knowledge base is configured to convert the original task description text in the atomic task package into vector query conditions, perform semantic similarity matching in the vector database, obtain matching text fragments related to the original task description text, and retain the original task description text. The large language model, based on matching text fragments, original task description text, and test case generation requirements, calls the test case generator interface through the MCP protocol to extract parameters from the test parameter set and generate structured test cases that conform to the predefined data structure. The automated testing framework is configured to invoke and execute structured test cases; The test case storage module is configured to store structured test cases; The construction of the RAG knowledge base involves segmenting and vectorizing the test requirement document, design document, test outline, and test process, storing them in a vector database, and preparing the test parameter set, which is also stored in the vector database. The document processing module cleans and formats the test requirement documents, design documents, test outlines and test procedures uploaded by users, removing special symbols, garbled characters and repeated paragraphs, retaining valid information and converting them into a uniform format. The text segmentation module splits documents whose content exceeds the context length of the large language model into text fragments, and sets the overlap of the fragment contexts to obtain the segmented knowledge fragments. The vectorization module converts the segmented knowledge fragments into high-dimensional vectors using the Embedding model, thus obtaining the vectorized text fragments. The vectorized text fragments are stored in the vector database and an index is created using the vector storage module. The process involves calling the RAG retrieval interface to convert the original task description text in the atomic task package into vector query conditions in the RAG knowledge base, performing semantic similarity matching in the vector database, obtaining matching text fragments related to the original task description text, and retaining the original task description text, including: The original task description text is presented in the form of natural language text, including the test objectives, test scope, and expected results; The retrieval module performs semantic similarity matching on the vector query conditions in the vector database to obtain the similarity between the vector query conditions and the vectorized text fragments in the vector database. The retrieval module filters out vectorized text fragments with high similarity based on a pre-set similarity threshold and uses them as matching text fragments related to the original task description text. The large language model, based on matching text fragments, original task description text, and test case generation requirements, calls the test case generator interface via the MCP protocol to extract parameters from the test parameter set and generate structured test cases conforming to a predefined data structure, including: The large language model receives the matching text fragments returned by the retrieval module and converts the original task description text and the information from the tools registered in the MCP adapter into vectors respectively. Obtain the similarity between two vectors, perform matching based on the obtained similarity, filter tools with similarity higher than the threshold from the toolchain list provided in the MCP adapter, and determine the interface of the test case generator; The test case generator's interface is called via the MCP protocol to extract parameters from the test parameter set and generate structured test cases that conform to a predefined data structure.

6. A structured test case generation server based on RAG and MCP, characterized in that, Including memory and processor; The memory is used to store computer-executable instructions; The processor is configured to execute the computer-executable instructions to implement the method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable instructions, which, when executed by a computer, enable the implementation of the method as described in any one of claims 1-4.

Images