Aviation equipment test case construction method based on RAG framework and Agenen architecture

Abstract

The invention discloses an aviation equipment test case construction method based on an RAG framework and an Aen architecture, and the method comprises the following steps: S1) collecting the historical test data and flow of aviation electromechanical equipment, and constructing a local knowledge base; s2) retrieving a test case similar to the current test requirement from a local knowledge base; s3) aiming at the long text document, constructing a knowledge graph by adopting a mode of extracting an abstract and a theme; the method comprises the following steps: for a large table document, carrying out long-to-short processing on table data by adopting a digital index mode; and S4) transmitting the retrieved information to the generative model, and in combination with the historical test case, calling and referring to the existing test data by using an RAG method under the Agent architecture to generate a new test case. According to the method, efficient understanding and processing of the long text can be achieved, existing test data are fully called and referred to, new test cases are generated, and therefore automation of the whole process is achieved.

Description

Technical Field

[0001] The present invention relates to a method for constructing a test case for aviation equipment, and in particular to a method for constructing a test case for aviation equipment based on a RAG framework and an Agen architecture. Background Art

[0002] In the aviation field, aviation equipment development involves a complete product life cycle management process, from product design, quality inspection during manufacturing, performance testing in various environments, to the final assembly process of system and component integration. The coordination and integration of these steps ensure the efficient and safe operation of aviation equipment throughout its life cycle.

[0003] MBSETest intelligent testing industrial software (hereinafter referred to as MBSETest software) is a model-based system process used in testing and evaluation. MBSE (Model-Based System Engineering) uses system models instead of traditional documentation methods for engineering management to cope with complex system engineering development. This method is called "Model-Based Testing and Evaluation System" in testing, which can help design, execute and optimize the testing process.

[0004] MBSETest software can realize rapid and agile development of test software, support convenient design of test cases, hardware-in-the-loop simulation, automatic code generation and integrated verification, and has the functions of remote control, collaborative operation, real-time monitoring, data analysis and auxiliary fault diagnosis of distributed intelligent testing and detection equipment. When using MBSETest software, it is necessary to extract and analyze requirements from the system requirements document to ensure that all functional and performance requirements are fully understood. In the traditional method, system modeling is performed after the requirements analysis, that is, a system modeling tool (such as SysML) is used to create a model of the system. These models include the structure, behavior and interface of the system, etc., in order to provide a basis for testing.

[0005] Functional decomposition: decompose the system's functions and identify key functional use cases and scenarios to ensure that test cases cover all aspects.

[0006] Test case design, design specific test cases based on functional use cases. Each test case should include input conditions, expected results, and execution steps for effective verification.

[0007] Automated test integration, the designed test cases are integrated with the test framework, and are first executed automatically to improve test efficiency.

[0008] Verification and review, perform verification and review to ensure that the test cases are complete and meet system requirements, and can effectively test the various functions of the system.

[0009] In recent years, with the development of deep neural networks, especially the introduction of the Transformer architecture, the development of large artificial intelligence models has been promoted. The emergence of large language models has made intelligent automation in the aviation field possible. However, large language models are essentially trained based on static data, and have limitations such as data bias and slow knowledge update. How to use artificial intelligence methods to automatically build test cases for aviation equipment is particularly urgent. RAG (Retrieval-Augmented Generation) is an artificial intelligence technology that combines information retrieval technology with language generation models. This technology retrieves relevant information from an external knowledge base and inputs it as a prompt to a large language model (LLMs) to enhance the model's ability to handle knowledge-intensive tasks, such as question answering, text summarization, content generation, etc., providing new possibilities for the automatic construction of aviation equipment test cases.

[0010] In addition, test case data in the aviation field usually exists in the form of large tables and long texts, containing a large amount of detailed information and professional knowledge. The complexity of the content and the huge amount of information make large-scale training of language models often face the problems of insufficient context understanding and information loss. Therefore, processing long texts and large tables is particularly necessary. RAG technology not only ensures the integrity of data information, but also significantly improves processing speed and accuracy through efficient understanding and processing of these long text data. Summary of the invention

[0011] The technical problem to be solved by the present invention is to provide an aviation equipment test case construction method based on the RAG framework and the Agen architecture, which can achieve efficient understanding and processing of long texts, generate answers with factual basis, fully call and refer to existing test data, generate new test cases, and thus realize the automation of the entire process.

[0012] The technical solution adopted by the present invention to solve the above technical problems is to provide an aviation equipment test case construction method based on the RAG framework and the Agen architecture, including the following steps: S1) collecting historical test data and processes of aviation electromechanical equipment to build a local knowledge base; S2) retrieving test cases similar to current test requirements from the local knowledge base; based on the RAG framework during retrieval, the local knowledge document is parsed into long text documents and large table documents; S3) for long text documents, a knowledge graph is constructed by extracting summaries and topics; for large table documents, a digital index is used to perform long-to-short processing on the table data; S4) the retrieved information is passed to the generation model, and the generation model uses this information to generate context-related answers or content; combined with historical test cases, a new test case is generated by calling the RAG method under the Agent architecture and referring to the existing test data.

[0013] Furthermore, the local knowledge base in step S1 includes a historical test case library, a test module library, a test process library and a board management library.

[0014] Furthermore, step S1 includes establishing a graph database around the test equipment, storing the test resources required for each test equipment in the graph database, the test resources including a board configuration table and a product wiring table, and step S2 is based on the logical thinking chain of the test process, using a combination of text database and graph database to retrieve similar historical test cases.

[0015] Furthermore, the step S2 designs components and test cases for the test program of the smallest unit, and divides the data flow into a sending module and a receiving module, then describes each unit module and assigns a separate ID, and then connects the required unit modules in series into the entire test case.

[0016] Furthermore, the step S1 also includes converting the information about pictures and videos in the historical aviation test data into text, adding index links, and expanding it into the graph database.

[0017] Furthermore, it also includes creating new test cases for MBSETest software under the Agent architecture in the following manner: extracting test modules from historical test cases and processes, summarizing relevant test objects based on historical test records, and recording commonly used test cases for each test object; developing prompt and function parts, designing similar test cases and test processes retrieved by RAG in the prompt, and developing action to call the function; calling the large language model, and inputting the prompt to obtain the created test case.

[0018] Furthermore, it also includes the step of adding think in Agent to RAG, vectorizing the KnowledgeGraph and storing it in the local knowledge base, searching for knowledge similar to the user's question according to the vector retrieval method, and then answering it.

[0019] Furthermore, it also includes using Agent to control each functional interface in the MBSETest software; when creating a new test case, using historical data to retrieve relevant knowledge, thinking about the matching degree between the relevant knowledge and user needs, and other knowledge required to meet user needs, performing multiple retrieval and thinking processes, and finally giving a summary answer of the reasoning content; and providing historical pictures or videos as a reference for new test cases.

[0020] Compared with the prior art, the present invention has the following beneficial effects: the aviation equipment test case construction method based on the RAG framework and the Agen architecture provided by the present invention realizes efficient understanding and processing of long texts through the combination of the retrieval module and the generation module; combined with historical test cases, the Agent uses the RAG method to call and refer to existing test data to generate new test cases, thereby realizing the automation of the entire process. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Constructing a flow chart for the aviation equipment test case of the present invention; Figure 2 Create a schematic diagram for the new test case based on the AI ​​Agent architecture of the present invention; Figure 3 A schematic diagram of establishing a graph database around a test device for the present invention; Figure 4 This is a schematic diagram of the steps of adding think in Agent to RAG in the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0023] Figure 1 Construct a flow chart for the aviation equipment test case of the present invention.

[0024] RAG (Retrieval Enhanced Generation) is a natural language processing model that combines retrieval and generation technologies. It generates factual answers by retrieving real content from the local knowledge base and combining it with a large language model, thereby making up for the shortcomings of the large language model in knowledge updating and accuracy. The solution for long-term processing based on RAG technology proposed in the present invention realizes efficient understanding and processing of long texts through the combination of retrieval module and generation module. Long-term mainly refers to large tables and long text data. When retrieving tables, RAG technology is limited by the input length of the input Embedding model and the input length of the large model. It is necessary to slice the larger table and text information, which can greatly ensure the integrity and accuracy of a document. The processing of long-term mainly includes table data and text data. For table data, the present invention uses a digital index to perform long-to-short processing on table data; for text data, the present invention uses the method of extracting abstracts and topics to construct a knowledge graph. Then, the long-term is converted into a short-term and then retrieved, which maintains the integrity of the data information.

[0025] Agent technology is also applied in the present invention. AI Agent is an artificial intelligence system with perception, decision-making and action capabilities, and can complete complex tasks autonomously. In the present invention, Agent is used to automatically build test cases. First, the object to be tested and the test steps are analyzed according to the test requirements, the test process is decomposed, the corresponding test cases are established, and resources are provided for the test. Combined with historical test cases, Agent uses the RAG method to call and refer to existing test data to generate new test cases, thereby realizing the automation of the entire process.

[0026] In the application of MBSETest, local documents or information are first retrieved from an external knowledge base, and the retrieved information is then passed to the generative model, which uses this information to generate contextually relevant answers or content. By combining external data with the generation process, RAG can significantly improve the accuracy and relevance of answers and reduce the possibility of generating false information.

[0027] Based on the RAG principle, historical test data and processes of aviation electromechanical equipment are collected.

[0028] 1. Design local knowledge bases, such as historical test case libraries, test module libraries, test process libraries, board management libraries, etc.

[0029] 2. According to the current test requirements, retrieve test cases similar to the current test requirements from the local knowledge base (such as the historical test case library). In this way, you can get inspiration and reference from the existing test data.

[0030] 3. Input the retrieved relevant test cases into the RAG framework, and generate new test cases based on the current system model and requirements. This step requires the combination of large language models and Agent technology.

[0031] AI Agent refers to an artificial intelligence system that can perform specific tasks autonomously. It uses artificial intelligence algorithms and models to create entities that can perform specific tasks, make decisions, and interact with the environment autonomously or semi-autonomously. It can understand, analyze, and respond to information input by the user. AI Agent is mainly composed of 4 parts: 1. Perception: that is, collecting external data input, etc.

[0032] 2. Information processing: It can be said that it is the use of large models combined with knowledge bases to process information.

[0033] 3. Execution: Based on a series of operations of the big model, draw conclusions, make plans for each step, and then execute them.

[0034] 4. Output: After execution, an output is required to inform the execution result.

[0035] Please continue to see Figure 2 According to the above AI Agent architecture, for the creation of new test cases for MBSETest software, the present invention performs the following operations: 1. First, extract the test modules from the historical test cases and processes. For example, in the avionics recording and loading system, summarize the relevant test objects based on the historical test records, and for each test object, the commonly used test cases.

[0036] 2. Develop the prompt and function parts, design similar test cases and test processes retrieved by RAG in the prompt, and develop the action to call the function.

[0037] 3. Call the large language model and enter the prompt to get the created test case.

[0038] In the Agent framework, tasks are first split according to the test requirements proposed by the user. For example, in the test system, test equipment and test functions are displayed. Then, RAG technology is used to search based on the identified entities.

[0039] In the common RAG solution, the local knowledge base is stored in text documents such as word, pdf, and txt, and knowledge is obtained through document parsing tools such as langchain. The present invention provides a logical thinking chain based on the test process, and uses a combination of text database and graph database to retrieve similar historical test cases.

[0040] In the local text data, design the data in the thinking chain format according to the test process. Design components and test cases for the smallest unit test program, such as ICD (Interface Control Documents) signal component 422 protocol, 429 protocol, etc. The data flow is divided into a sending module and a receiving module, and then describe each unit module and assign a separate ID, and then use these unit modules to connect in series to form the entire test case.

[0041] In the whole process of testing, different test resources are required according to different test equipment. Test resources include board configuration table, product wiring table, etc. Build a graph database around the test equipment, such as Figure 3 As shown: Establish each functional interface for MBSETest software, and develop functions to realize the linkage between the large language model and MBSETest by inputting parameters and outputting functions. Use Agent framework to establish ReAct to call function program and realize the functions in the software.

[0042] In the historical aviation test data, there are materials about pictures and videos. These materials are incomprehensible to the large language model, but considering that the present invention requires multi-dimensional retrieval and analysis, the present invention converts pictures and videos into text, adds index links, and expands them into the graph database. The present invention summarizes pictures and videos into summary text, and converts pictures and videos into local links for front-end calls.

[0043] The present invention is different from the previous combination of RAG and KnowledgeGraph. The think step in Agent is added to RAG. That is, in traditional RAG and KnowledgeGraph, the KnowledgeGraph is vectorized and stored in the local knowledge base, and the knowledge similar to the user's question is searched according to the vector retrieval method, and then the answer is given. In the present invention, the idea of ​​think is introduced. After the relevant knowledge is queried, the big model is asked to perform the task of think. What other knowledge is needed to solve the user's problem? The required knowledge is searched again. Finally, all the retrieved knowledge is integrated and the big model is asked to answer. The specific process is as follows: Figure 4 shown.

[0044] In this process, the present invention not only uses RAG technology to refer to the local map knowledge base, but also uses the Agent solution, adds task decomposition, tool calling and other methods, conducts deep mining and matching of the local knowledge base, finds the most relevant knowledge, and gives accurate answers. Through the link records in the knowledge base, the large model can output picture or video links, thereby realizing the interaction of pictures and videos.

[0045] In MBSETest software, the present invention uses Agent to control each functional interface in the software. In creating a new test case, relevant knowledge is retrieved using historical data, and the matching degree between the relevant knowledge and user needs is considered, as well as other knowledge required to meet user needs. Multiple retrieval and thinking processes are performed, and finally the reasoning content summary answer is given. In this process, historical pictures or videos can be given as references for new test cases.

[0046] In summary, the present invention not only improves the performance of large models, but also expands their application scope, enabling them to better meet the needs of users and bring users a more convenient and intelligent experience. The specific advantages are as follows: 1. The combination of RAG and Agent opens up new possibilities for the application of large models. Through deep search and matching, combined with the local map knowledge base, the present invention can achieve more accurate and efficient information retrieval. In addition, the introduction of local links of pictures and videos makes the large model no longer limited to text interaction, but can provide a richer and more intuitive information experience.

[0047] 2. The use of RAG technology ensures the accuracy of the big model in providing answers, reducing the occurrence of "hallucination" phenomena, that is, the generation of erroneous or fictitious information. At the same time, the application of Agent technology allows the big model to handle complex tasks more flexibly, and provide users with more comprehensive and personalized services through task decomposition and calling external tools.

[0048] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. A method for constructing aviation equipment test cases based on RAG framework and Agen architecture, characterized in that: The steps include: S1) Collect historical test data and processes of aviation electromechanical equipment and build a local knowledge base; S2) Retrieve test cases similar to the current test requirements from the local knowledge base; based on the RAG framework, the local knowledge documents are parsed into long text documents and large table documents; S3) For long text documents, we extract summaries and topics to build knowledge graphs; for large table documents, we use digital indexing to process table data from long to short. S4) The retrieved information is passed to the generative model, which uses the information to generate context-related answers or content; combined with historical test cases, the RAG method is used under the Agent architecture to call and refer to existing test data to generate new test cases.

2. The method for constructing aviation equipment test cases based on the RAG framework and the Agen architecture as claimed in claim 1, characterized in that: The local knowledge base in step S1 includes a historical test case library, a test module library, a test process library and a board management library.

3. The method for constructing aviation equipment test cases based on the RAG framework and the Agen architecture as claimed in claim 1, characterized in that: The step S1 includes establishing a graph database around the test equipment and storing the test resources required for each test equipment in the graph database, wherein the test resources include a board configuration table and a product wiring table. The step S2 is based on the logical thinking chain of the test process and uses a combination of a text database and a graph database to retrieve similar historical test cases.

4. The method for constructing aviation equipment test cases based on the RAG framework and the Agen architecture as claimed in claim 3, characterized in that: The step S2 designs components and test cases for the test program of the smallest unit, divides the data stream into a sending module and a receiving module, then describes each unit module and assigns a separate ID, and then connects the required unit modules in series into the entire test case.

5. The method for constructing aviation equipment test cases based on the RAG framework and the Agen architecture as claimed in claim 3, characterized in that: The step S1 also includes converting the information about pictures and videos in the historical aviation test data into text, adding index links, and expanding it into the graph database.

6. The method for constructing aviation equipment test cases based on the RAG framework and the Agen architecture as claimed in claim 1, characterized in that: It also includes creating new test cases for MBSETest software under the Agent architecture as follows: Extract test modules from historical test cases and processes, summarize relevant test objects based on historical test records, and record common test cases for each test object; Develop prompt and function parts, design similar test cases and test processes retrieved by RAG in prompt, and develop action to call function; Call the large language model and enter the prompt to get the created test case.

7. The method for constructing aviation equipment test cases based on the RAG framework and the Agen architecture as claimed in claim 6, characterized in that: It also includes the steps of adding think in Agent to RAG, vectorizing the KnowledgeGraph and storing it in the local knowledge base, searching for knowledge similar to the user's question according to the vector retrieval method, and then answering it.

8. The method for constructing aviation equipment test cases based on the RAG framework and the Agen architecture as claimed in claim 6, characterized in that: It also includes using Agent to control each functional interface in the MBSETest software; when creating new test cases, using historical data to retrieve relevant knowledge, thinking about the matching degree between relevant knowledge and user needs, and other knowledge required to meet user needs, performing multiple retrieval and thinking processes, and finally giving a summary answer to the reasoning content; Historical pictures or videos can also be provided as references for new test cases.

Images