A test case automatic generation method based on enhanced RAG

Abstract

The present invention relates to the field of artificial intelligence technology, and provides a method for automatically generating test cases based on enhanced RAG, including extracting text from system documents, and dividing the extracted text into text segments to form text vectors, which are stored in a vector database; a local first language model generates corresponding triples, which are stored in a knowledge graph database; the system documents are directly handed over to the local first language model, and the article outline is extracted and stored in a relational database; according to the test case requirements input by the user, text segments, triples and article outlines related to the test case generation are obtained and sent to the local second language model; the second language model automatically generates specific test cases and detailed test case documents based on this information. The present invention can realize the automatic generation of test case books to reduce the work content and improve work efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a test case automatic generation method based on enhanced RAG. Background Art

[0002] In the traditional system testing process, testers manually write outline test cases and execute them, and finally complete the test report.

[0003] RAG (Retrieval Augmented Generation) technology is a natural language processing technology that combines retrieval and generation functions. It assists the generation model to answer questions by retrieving information related to the input question from a large external database. The existing RAG technology can give all the information of the system, such as the requirements definition document, system design document, test report template, etc., as a knowledge base to AI for learning. Testers can use AI to automatically generate outline use cases, etc.

[0004] However, the defect is that the RAG technology of the prior art only divides the data into segments and then matches them with the user's input. The answer obtained through such a single-way recall is very limited in scope and the words are generalized, and only 20% are usable. Therefore, the present invention provides a three-way recall enhanced RAG model to improve the availability of generated test cases. After replacement, the availability reaches more than 90%. Summary of the invention

[0005] The present invention mainly solves the technical problems that the test cases generated by RAG in the prior art are limited in scope and use generalized terms, and proposes a test case automatic generation method based on enhanced RAG to realize the automatic generation of test case books, so as to reduce the work content and improve work efficiency.

[0006] The present invention provides a method for automatically generating test cases based on enhanced RAG, comprising:

[0007] Extract text from system documents, and divide the extracted text into text segments to form text vectors, which are stored in the vector database;

[0008] The segmented text segments are passed to the local largest language model to generate corresponding triples and store them in the knowledge graph database;

[0009] The system documents are directly handed over to the local No. 1 language model to extract the article outline and store it in a relational database;

[0010] According to the test case requirements input by the user, search is performed from the vector database, knowledge graph database and relational database. The cosine similarity algorithm is used to obtain text fragments related to test case generation from the vector database, and then the text fragments are sorted according to similarity using the re-ranking model; the entity keyword similarity is used to match the corresponding triples from the knowledge graph database; and the corresponding article outline is directly obtained from the relational database;

[0011] The matched text fragments, triples and article outlines related to test case generation are sent to the local second largest language model; the second largest language model automatically generates specific test cases and detailed test case documents based on this information.

[0012] Furthermore, the system documents include but are not limited to requirement definition documents, system design documents, test report templates,

[0013] Furthermore, a text segmentation tool is used to segment the system documents into text segments;

[0014] Use embedding models to convert text snippets into text vectors.

[0015] Furthermore, the first language model extracts entity keywords in the segment through the created prompt word template and generates corresponding triples.

[0016] Furthermore, the system documents are directly handed over to the local No. 1 language model to extract the article outline and store it in a relational database, including:

[0017] The first language model traverses the pages of the system document, extracts the outline content of each page, and finally summarizes the outline of each page. The first language model summarizes it again to obtain the overall outline and stores it in the relational database.

[0018] Furthermore, the rearrangement model uses a cosine similarity algorithm to calculate the distance between the user input and the matched text segment, thereby achieving similarity sorting.

[0019] The present invention provides a method for automatically generating test cases based on enhanced RAG, which performs text segmentation on system documents and converts them into text vectors and stores them in a vector database. The system documents are handed over to the local first largest language model to generate triples, which are stored in a knowledge graph database; the system documents are handed over to the local first largest language model to extract the article outline, which is stored in a relational database; and the user's input is matched with the three databases to obtain an answer. The present invention adopts a unique three-way recall enhanced RAG model, and the data engineer uploads the relevant system documents of the system to be tested in advance. When there is a need to generate test cases, the tester sends the part to be tested to the local second largest language model. At this time, the relevant system documents uploaded by the data engineer have been processed into three types of data by the three-way recall enhanced RAG, which are matched with the tester's input through code logic to obtain the most relevant part, and then all are sent to the local second largest language model. Relying on the powerful natural language processing capability of the local second largest language model, test cases can be automatically generated, and then the test case book template is written through code logic to realize the automatic generation of test case books. The original testers needed to write outlines, write test cases, execute test cases, and write test case books. After using the technical solution of this application, 70% of the work is saved. After executing the test cases, the results can be written into the test case book, which greatly reduces the work content and improves work efficiency. Compared with ordinary RAG, the availability of test case results generated by the technology of this application can reach more than 90%. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is an implementation flow chart of the method for automatically generating test cases based on enhanced RAG provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the technical problems solved by the present invention, the technical solutions adopted and the technical effects achieved clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all the contents.

[0022] like Figure 1 As shown, an embodiment of the present invention provides a method for automatically generating test cases based on enhanced RAG, including the following process:

[0023] Step 1: extract text from system documents and divide the extracted text into text segments to form text vectors, which are stored in a vector database.

[0024] The system documents include but are not limited to requirement definition documents, system design documents, and test report templates.

[0025] The present invention uses a text segmentation tool to segment system documents into text fragments. The text segmentation tool provided by the present invention is a comprehensive text processing tool that combines multiple advanced technologies such as OCR (optical character recognition), layout detection, formula recognition and table recognition, and is intended to efficiently extract and segment text content from various documents. First, the text content in the document is identified by OCR technology to ensure high-accuracy text extraction. Secondly, the visual layout of the document is analyzed using layout detection technology to identify the position and hierarchy of different text blocks, including paragraphs, titles, headers, footers, etc. In addition, the tool also has a formula recognition function that can accurately extract and parse complex mathematical expressions, as well as a table recognition function that recognizes the row and column structure of the table and extracts text data in the cell. In the text extraction stage, the tool combines the results of OCR, layout detection, formula recognition and table recognition to extract all text information in the document. Finally, the extracted text is segmented using natural language processing (NLP) technology, and the text is further segmented into segments suitable for processing according to the token length of the sentence to generate structured text segments for subsequent analysis and processing.

[0026] Afterwards, the text fragments are converted into text vectors using an embedding model (after the embedding model segments the text into words, it uses vocabulary and vector mapping to achieve the conversion of text into high-dimensional text vectors) and stored in a vector database.

[0027] Step 2: Submit the segmented text fragments to the local largest language model to generate corresponding triples and store them in the knowledge graph database;

[0028] The first language model extracts entity keywords from the fragments through the created prompt word template and generates corresponding triples, i.e., knowledge graph data. The triples include subjects (people, organizations, or objects), relationships, and objects (people, places, or objects).

[0029] The prompt word template is, for example: Given the following text, please analyze and extract the relationship triples in it. Each triple should include a subject (person, organization or object), a relationship and an object (person, place or object). If there is no obvious relationship in the text, please return an empty string. \n\nText: "{content}"\n\nPlease extract the relationship triple list in the following format:\n-(subject, relationship, object)\n-(subject, relationship, object)\n\nIf there is no identifiable relationship, please return an empty string.

[0030] Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence.

[0031] Step 3: Submit the system document directly to the local first language model, extract the article outline, and store it in the relational database.

[0032] In this step, the first language model first traverses the pages of the system document, extracts the outline content of each page, and finally summarizes the outline of each page. The first language model then summarizes and obtains the overall outline, which is stored in the relational database.

[0033] Based on the above scheme, steps 1 to 3 can be performed simultaneously or sequentially.

[0034] Step 4: According to the test case requirements input by the user, search from the vector database, knowledge graph database and relational database, use the cosine similarity algorithm to obtain text fragments related to test case generation from the vector database, and then use the rearrangement model to sort the text fragments according to similarity; use the entity keyword similarity from the knowledge graph database to match the corresponding triples; directly obtain the corresponding article outline from the relational database. The rearrangement model is pre-trained with a large amount of data assets. The rearrangement model uses the cosine similarity algorithm to calculate the distance between the user input and the matched text fragment, so as to achieve similarity sorting.

[0035] This step extracts text snippets related to test case generation from the test case requirements. These snippets usually contain information such as functional descriptions, user scenarios, and expected behaviors. Then, key triples are extracted from these text snippets. These triples describe the key entities, actions, and goals in the system, providing basic data for generating test cases. Subsequently, an article outline is automatically constructed to help organize the structure and content of the test cases.

[0036] Step 5: Send the matched text fragments, triples, and article outlines related to test case generation to the local second largest language model; the second largest language model automatically generates specific test cases and detailed test case documents based on this information.

[0037] This step sends the matched text fragments, triples, and article outlines to the local second largest language model. The second largest language model automatically generates specific test cases based on this information, including test steps, expected results, preconditions, etc. By combining the input information, the second largest language model can generate detailed test case documents to ensure that the test cases are consistent with the descriptions in the requirements document and cover all key functional points and user scenarios.

[0038] After receiving the instruction, the local second largest language model responds according to the requirements of the answer instruction template.

[0039] The answer instruction template is, for example:

[0040] <User input>{{question}}<User input>\n

[0041] <Known information>{{content}}<Known information>\n

[0042] <test case structure>{{structure}}<test case structure>

[0043] Requirements: Please generate a clearly structured test case according to <Test case structure>.

[0044] The user's input is the user's demand, and the known information is the specific data matched by the present invention.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that modifying the technical solutions described in the aforementioned embodiments, or replacing some or all of the technical features therein by equivalents, does not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A test case automatic generation method based on enhanced RAG, characterized in that: include: Extract text from system documents, and divide the extracted text into text segments to form text vectors, which are stored in the vector database; The segmented text segments are passed to the local largest language model to generate corresponding triples and store them in the knowledge graph database; The system documents are directly handed over to the local No. 1 language model to extract the article outline and store it in a relational database; According to the test case requirements input by the user, search is performed from the vector database, knowledge graph database and relational database. The cosine similarity algorithm is used to obtain text fragments related to test case generation from the vector database, and then the text fragments related to test case generation are sorted according to similarity using the rearrangement model; the entity keyword similarity is used to match the corresponding triples from the knowledge graph database; the corresponding article outline is directly obtained from the relational database; Send the matched text snippets, triples, and article outlines related to test case generation to the local second largest language model; The second largest language model automatically generates specific test cases and detailed test case documents based on the matched text fragments, triples, and article outlines related to test case generation.

2. The method for automatically generating test cases based on enhanced RAG according to claim 1, characterized in that: The system documents include but are not limited to requirement definition documents, system design documents, and test report templates.

3. The method for automatically generating test cases based on enhanced RAG according to claim 1, characterized in that: Use text segmentation tools to segment system documents into text segments; Use embedding models to convert text snippets into text vectors.

4. The method for automatically generating test cases based on enhanced RAG according to claim 3, characterized in that: The first language model extracts entity keywords in the fragment and generates corresponding triples by creating a good prompt word template.

5. The method for automatically generating test cases based on enhanced RAG according to claim 4, characterized in that: The system documents are directly handed over to the local No. 1 language model to extract the article outline and store it in a relational database, including: The first language model traverses the pages of the system document, extracts the outline content of each page, and finally summarizes the outline of each page. The first language model summarizes it again to obtain the overall outline and stores it in the relational database.

6. The method for automatically generating test cases based on enhanced RAG according to claim 5, characterized in that: The rearrangement model uses a cosine similarity algorithm to calculate the distance between the user input and the matched text segment, thereby achieving similarity sorting.

Images