Test case generation method and system based on multi-agent efficient collaboration

Abstract

The invention relates to a test case generation method and system based on multi-agent efficient collaboration, belongs to the technical field of software testing, and solves the problems of incomplete scene coverage, logic disorder and the like when a single large model processes a complex task. The method comprises the following steps: constructing a multi-modal professional field knowledge base; the task planning agent module generates a test case generation task based on a project development document and a software source code file of a to-be-tested project and distributes the test case generation task to the test demand analysis agent module; based on the test case generation task, extracting a test demand point related to the tested software configuration item, describing the test demand point, and labeling a corresponding tracking relationship between a test demand point ID and a code snippet or a function in the software source code file to construct a test demand-code snippet / function set; and generating a test case and a test description document based on the test demand-code snippet / function set and the multi-modal professional domain knowledge base. And the ability of generating the test case of the large model is enhanced through organic and efficient cooperation of multiple agents.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, in particular to a test case generation method and system based on efficient cooperation of multiple agents. BACKGROUND

[0002] Software testing is a key link to ensure software quality and reliability, especially in the fields of aviation, aerospace and other high safety and stability requirements, its role is particularly prominent. In these high-risk application scenarios, software failure can have serious consequences, so it is necessary to fully verify the software through systematic testing means. The testing process is designed and implemented to cover a comprehensive test case, aiming to find defects and potential faults in the software, and to verify whether it meets the established requirements specifications in terms of function implementation, logical correctness and exception handling. This not only relates to whether the software can run as expected, but also directly affects the overall credibility and mission support capability of the system.

[0003] The traditional test case design is highly dependent on the personal experience and subjective judgment of the test personnel, from the understanding of development documents (such as software requirements specification, software design specification, development general requirements, technical scheme, etc.) and software code to the preparation of test outline and test case generation, there is a lack of systematic and standardized methods and rapid and efficient knowledge reuse mechanism, resulting in low efficiency, large resource investment and high cost in the test case design process. At the same time, this mode is difficult to effectively integrate historical test data, typical defect library and domain expert experience, resulting in insufficient knowledge asset deposition and poor reusability. The quality of the test cases generated by this mode is unstable, and there are generally coverage blind spots, logical redundancy, repeated testing or key path omission, etc., which brings high quality risk to software testing work.

[0004] In recent years, the rapid development of artificial intelligence technology, especially large language models (LLM), has provided a new technical path for the intelligent transformation of software testing. With its outstanding ability in natural language understanding and generation, large language models can deeply understand unstructured text such as requirement documents and design specifications, supporting higher degree of automation in key links such as test requirement analysis, test outline preparation and test case generation, and providing more scalable technical support for high-reliability systems and other fields with strict requirements for test sufficiency. However, the test case automatic generation scheme based on large models still faces certain limitations in practical application, mainly in the following aspects: (1) Lack of professional domain knowledge: existing large models are mostly trained based on general corpus, lacking in-depth understanding of the terminology system, design specifications and business processes of specific professional fields (such as avionics, industrial control, etc.), resulting in deviations in semantic accuracy and scenario applicability of the generated test cases.

[0005] (2) Model hallucination problem is prominent: when facing incomplete or ambiguous input, the model tends to "guess" and generate seemingly reasonable but actually incorrect or non-existent test logic, resulting in a large number of invalid or misleading use cases, increasing the cost of manual screening and affecting the credibility of testing.

[0006] (3) Output redundancy and repetition are serious: the model is prone to generate test cases with similar semantics, similar operation paths, or overlapping coverage targets during the generation process, resulting in test set inflation, which not only reduces execution efficiency but also increases the invalid consumption of test resources.

[0007] (4) Test scene coverage is incomplete: when dealing with long-length requirement documents, cross-module interaction logic, or multi-level state transition scenarios, the model is prone to miss key information or misjudge logic, miss edge scenarios, lack performance scenarios, and incomplete abnormal scenario coverage, making it difficult to accurately restore the full picture of system behavior, resulting in incomplete test coverage or distorted scenarios.

[0008] (5) Lack of result interpretability: the generation process of test cases lacks transparency, lacks explicit reasoning path and basis tracing mechanism, and is difficult to support quality review, compliance audit and responsibility definition, which restricts its feasibility in high-integrity and high-security level systems. SUMMARY

[0009] In view of the above analysis, the embodiments of the present application aim to provide a test case generation method and system based on efficient cooperation of multiple agents to solve the problems of insufficient reuse of historical knowledge assets, lack of professional knowledge of large models, prominent hallucination problem, serious output redundancy and repetition, incomplete test scene coverage, and lack of result interpretability.

[0010] On the one hand, the embodiments of the present application provide a test case generation method based on efficient cooperation of multiple agents, comprising: hierarchically classifying and screening project development documents, software source code files, project test documents, software test document templates, expert review opinions, national / industry standards and typical defects of historical projects and normalizing processing to construct a multi-modal professional knowledge base; generating a test case generation task based on the project development documents and software source code files of the to-be-tested project through a task planning agent and distributing it to a test requirement analysis agent; extracting test requirement points involved by the software under test configuration and their descriptions based on the test case generation task and labeling the corresponding tracking relationship between the test requirement point ID and the code snippet or function in the software source code file to construct a test requirement-code snippet / function set; generating test cases and test specification documents based on the test requirement-code snippet / function set and the multi-modal professional knowledge base.

[0011] The beneficial effects of the above technical solutions are as follows: by designing multiple intelligent agents with specific roles and functions, the ability of the large model to generate test cases based on development documents and software source code is enhanced. The professionalism and accuracy of test case generation are significantly improved, and the model illusion is effectively reduced. Through the organic integration of Multi-Agent and ReAct mode, the complex task is automatically decomposed and efficiently cooperated, and the efficiency and coverage are greatly improved.

[0012] In another aspect, the embodiment of the present application provides a test case generation system based on efficient cooperation of multiple agents, comprising: a multi-modal professional knowledge base construction module for hierarchical classification and screening and standardized processing of project development documents, software source code files, project test documents, software test document templates, expert review opinions, national / industry standards and typical defects of historical projects to construct a multi-modal professional knowledge base; a task planning intelligent agent module for generating a test case generation task based on the project development documents and software source code files of a to-be-tested project and distributing it to a test requirement analysis intelligent agent; a test requirement analysis intelligent agent for extracting test requirement points involved by a software configuration item under test and their descriptions based on the test case generation task and labeling the corresponding tracking relationship between the test requirement point ID and the code snippet or function in the software source code file to construct a test requirement-code snippet / function set; a test case generation intelligent agent module for generating test cases and test specification documents based on the test requirement-code snippet / function set and the multi-modal professional knowledge base.

[0013] Compared with the prior art, the present application can at least realize one of the following beneficial effects: 1. Significantly improve the professionalism and accuracy of test case generation, effectively reduce the model illusion. By constructing a high-quality hierarchical multi-modal professional knowledge base (test case text library, test case correlation graph library, software code library, test document template library, expert review knowledge base, national / industry standard library, typical defect library), and adopting a dynamic hybrid parallel retrieval strategy (combining vector semantic and keyword hybrid retrieval and graph relationship retrieval), accurate and authoritative professional knowledge enhancement (RAG) is provided for each large model intelligent agent. This mechanism greatly makes up for the short board of general large models in the specific software testing field, makes the generated test outline and test case content more accurate and standardized, reduces redundancy and repetition rate, and effectively suppresses the "illusion" problem caused by knowledge deficiency.

[0014] 2. The full-link bidirectional traceability of "test requirements-test items-test cases-software code" is realized, which greatly enhances the explainability and reliability of the results. The present application establishes accurate mapping and tracking relationships between test requirements, test items / test sub-items in the test tree graph, specific test cases, and software source code functions or fragments through unique identification ID and association graph technology. This design allows the source of any test case to be clearly traced and verified, which completely changes the lack of explainability caused by the traditional large model "black box" generation mode, and provides unprecedented transparency and reliability for test case review, debugging, and maintenance.

[0015] 3. Through the organic integration of Multi-Agent and ReAct mode, the automatic decomposition and efficient collaboration of complex tasks are realized, greatly improving efficiency and coverage. The system adopts special agents (Agents) based on ReAct (Reasoning and Acting) architecture for division of labor and cooperation (such as task planning, requirement analysis, test tree graph generation, test outline generation, test case generation, review, etc.), simulating the refined collaboration workflow of a team of human experts. Each agent can independently call tools and handle subtasks through the "reasoning-acting" cycle, and is scheduled in order through the workflow engine. The system can dynamically plan the number of tasks that can be carried out in parallel according to the current task complexity and system hardware resource conditions, realize optimal resource allocation and task parallel batch processing, and significantly improve the coverage of the test scenario and the overall test design efficiency, overcoming the defects of incomplete scene coverage and logical confusion when a single large model processes complex tasks.

[0016] 4. An interactive visual confirmation and iteration mechanism is introduced to build an efficient human-machine collaborative closed loop, ensuring the quality of the generated content while significantly reducing labor costs. The present application innovatively designs a visual human-machine interface centered on the test tree graph, presenting the intermediate results (such as test tree graphs) and final results (test case sets) generated by the agent to the test personnel in a graphical manner for review, confirmation, optimization, and completion. This mechanism combines the domain experience and high-level decision-making ability of humans with the automated generation capabilities of agents, forming an iterative optimization closed loop of "generation-review-correction-confirmation". This not only effectively guarantees the high quality of the final output, but also frees test experts from tedious test case writing work, allowing them to focus more on high-value design and review work, thereby improving test quality while reducing overall test costs.

[0017] 5、Based on dynamic prompt word engineering and continuous learning, a sustainable evolution test case generation system is constructed. The application designs a dynamic prompt word template library, and the agent can adaptively select and combine the prompt word template according to the current specific task context, and integrate the real-time professional knowledge retrieved from the knowledge base to form accurate and efficient dynamic prompt word instructions. In addition, through the knowledge management agent, the system can automatically deposit the new knowledge assets generated in the project practice (such as the test cases, test documents, software codes, expert review opinions, etc. that are approved) into the knowledge base after classification and labeling, realizing the self-learning and continuous evolution of the system. This makes the whole system have the ability of continuous evolution, and the generation quality of test cases is continuously improved with the enrichment of the knowledge base and the optimization of the prompt words, solving the problem of static system being easy to be outdated and degenerated.

[0018] In the application, the above technical solutions can be combined with each other to realize more preferred combination solutions. Other features and advantages of the application will be described in the subsequent description, and some advantages will become apparent from the description, or will be understood by implementing the application. The purposes and other advantages of the application can be realized and obtained from the contents specifically pointed out in the description and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application, and together with the description serve to explain the principles of the application, and should not be considered as limiting the scope of the application, in which like reference numerals refer to like parts throughout the accompanying drawings. Figure 1 Flow chart of the test case generation method based on multi-agent efficient collaboration according to the embodiment of the application; Figure 2 First-stage test tree diagram according to the embodiment of the application; Figure 3 Second-stage test tree diagram according to the embodiment of the application; Figure 4 System architecture diagram of the test case generation system based on multi-agent efficient collaboration according to the embodiment of the application; Figure 5 Workflow diagram of the test case generation system based on multi-agent efficient collaboration according to the embodiment of the application. DETAILED DESCRIPTION

[0020] The preferred embodiments of the application will be specifically described below with reference to the accompanying drawings, wherein the drawings constitute a part of this application, and are used to illustrate the principles of the embodiments of the application, and should not be considered as limiting the scope of the application.

[0021] REFERENCE Figure 1A specific embodiment of the present invention discloses a test case generation method based on efficient multi-agent collaboration, comprising the following steps: In step S101, the project development documents, software source code files, project test documents, software test document templates, expert review opinions, national / industry standards, and typical defects of historical projects are hierarchically classified, screened, and standardized to construct a multimodal professional domain knowledge base. Specifically, the hierarchical classification, screening, and standardization further include: a first-level classification based on the type of software being tested involved in the project, including embedded software, non-embedded software, and FPGA software; a second-level classification based on the professional domain of the tested object involved in the project, including satellite control systems, aircraft inertial navigation, and ship radar; a third-level classification based on the test type, including functional testing, performance testing, interface testing, and security testing; and the construction of a unique identifier ID system using a hierarchical combination coding method, followed by the generation of unique identifier IDs for bidirectional tracking of requirements, code, and use cases.

[0022] In step S102, the task planning agent generates test cases and tasks based on the project development documents and software source code files of the project to be tested, and distributes them to the test requirements analysis agent. Specifically, the process of generating test cases and distributing them to the test requirements analysis agent through a task planning agent based on the project development documents and software source code files of the project to be tested further includes: inputting the project development documents and software source code files of the project to be tested into the task planning agent through a human-computer interaction interface, wherein the project development documents include software requirements specifications, software design specifications, overall development requirements, and technical solutions; the task planning agent identifies the testers' intentions, then coordinates and plans the task implementation path and selects an appropriate workflow, and distributes the test case generation task to the test requirements analysis agent; wherein, the task planning agent coordinates and plans the task implementation path and the required agents for different task requirements, so as to schedule multiple agents to efficiently collaborate to complete the test requirements analysis and mining tasks, test tree diagram generation tasks, test outline generation tasks, test case generation tasks, test case quality assessment and reflection tasks, etc., and the multiple agents include the test requirements analysis agent, the test tree diagram generation agent, the test outline generation agent, the test case generation agent, the test case quality assessment and reflection agent, the test document automatic generation agent, and the knowledge management agent.

[0023] The MCP and A2A protocols are used to enable communication between different agents, between agents and multimodal domain knowledge bases, and between agents and tools.

[0024] In step S103, the test requirement points and their descriptions involved in the configuration items of the software under test are extracted based on the test case generation task, and the correspondence between the test requirement point ID and the code snippets or functions in the software source code file is marked to construct a test requirement-code snippet / function set.

[0025] In step S104, test cases and test specification documents are generated based on test requirements - code snippets / function sets and a multimodal domain knowledge base. Specifically, generating test cases and test specification documents based on test requirements - code snippets / function sets and a multimodal domain knowledge base further includes: generating a first-stage test tree diagram based on test requirements - code snippets / function sets; generating a final version of the test outline document based on the first test tree diagram; generating a second test tree diagram based on the first test tree diagram and the final version of the test outline document; generating all test cases based on the second test tree diagram and the software test outline, and then evaluating the quality of all test cases to generate a complete test case set; calling the test tree diagram to generate an agent to update the second test tree diagram based on the complete test case set; inputting the complete test case set and the updated second test tree diagram into the test document to automatically generate an agent, and calling the test case document template in the multimodal domain knowledge base to generate the test specification document.

[0026] The generation of the first-stage test tree diagram based on test requirements - code snippets / function sets further includes: the test tree diagram generation agent performs parallel retrieval of a multimodal professional domain knowledge base according to the test requirements - code snippets / function sets, wherein the multimodal professional domain knowledge base includes a test case text library, a test case association graph library, a software code library, and a typical defect library; the first retrieval results are weighted and fused and then input into a dynamic prompt word construction engine, which dynamically assembles the first complete prompt word according to the test tree diagram generation task of the project to be tested; the first complete prompt word is input into a large model to generate the corresponding test tree diagram, and the test tree diagram is reviewed, confirmed, optimized and completed before the first test tree diagram is generated.

[0027] The software test outline generation based on the first test tree graph further includes: The test outline generation agent performs task analysis and evaluation on the first test tree graph, dynamically plans the number of multi-threaded parallel batch generation tasks based on the number of test items and sub-items and the current hardware computing resources, and performs parallel retrieval of the multimodal professional domain knowledge base; the second retrieval results are weighted and fused and input into a dynamic prompt word construction engine, which dynamically assembles a second complete prompt word based on the test outline generation task of the project to be tested; the second complete prompt word is input into a large model to generate descriptions of each test item, each test sub-item, and the corresponding test method for each test type; the generated descriptions of each test item, each test sub-item, and the corresponding test method for each test type are optimized, completed, and reviewed, and then input into the test document automatic generation agent; the test document automatic generation agent calls the test outline document template in the multimodal professional domain knowledge base to generate a compliant test outline document, modifies and reviews the test outline document, and outputs the final test outline document.

[0028] The generation of the second test tree diagram, based on the first test tree diagram and the final test outline document, further includes: the test tree diagram generating agent performing parallel searches of the multimodal professional domain knowledge base according to the first test tree diagram and the final test outline document; inputting the weighted fusion of the third search results into the dynamic prompt word construction engine, which dynamically assembles the third complete prompt word according to the test tree diagram generation task of the project to be tested; inputting the third complete prompt word into the large model to generate a complete test tree diagram; and generating the second test tree diagram after reviewing, confirming, optimizing and completing the complete test tree diagram.

[0029] The generation of test cases based on the second test tree diagram and the final test outline document further includes: the test case generation agent performs task analysis and evaluation on the final test outline document and the second test tree diagram, dynamically plans the number of multi-threaded parallel batch generation tasks according to the number of test sub-items and the current hardware computing resources, and performs parallel retrieval of the multimodal professional domain knowledge base; the fourth retrieval result is weighted and fused and then input into the dynamic prompt word construction engine, which dynamically assembles the fourth complete prompt word according to the test case generation task of the project to be tested; the fourth complete prompt word is input into the large model to generate all test cases for each test sub-item corresponding to each test type.

[0030] The process of evaluating the quality of all test cases to generate a complete test case set further includes: displaying all test cases to testers through a human-computer interaction interface, subjectively evaluating the quality of test cases through tester scoring, whereby unqualified test cases generated by the large model are directly edited, modified, and reviewed through the human-computer interaction interface, or regenerated by the large model, and then the regenerated test cases are selected and reviewed; the test case quality evaluation and reflection agent calculates the test case adoption rate, requirement coverage, and executability, where the ratio of the number of directly adopted test cases and the number of modified and adopted test cases to the total number of generated test cases is used as the test case adoption rate; the ratio of the number of test requirement points that the generated test cases can trace to cover to the total number of test requirement points is used as the requirement coverage rate; the ratio of the number of generated test cases with directly executable conditions to the total number of test cases is used as the executability; and the evaluation results of the test case adoption rate, requirement coverage, and executability are weighted and fused by setting weights for the test case adoption rate, requirement coverage, and executability to objectively evaluate the quality of test cases and generate a complete test case set.

[0031] After generating test cases and test specification documents, the knowledge assets formed during the execution of the test case generation task are automatically classified, filtered, and subject to metadata standards and review by a knowledge management intelligent agent. Then, after manual review and confirmation through a human-computer interaction interface, they are stored in a multimodal professional domain knowledge base. The knowledge assets include project development documents of the project to be tested, software source code files, test requirements - code snippets / function sets, test cases, test outline documents, test specification documents, etc.

[0032] In the following text, refer to Figure 2 , Figure 3 and Figure 4 The steps of the test case generation method based on efficient multi-agent collaboration according to embodiments of the present invention will be described in detail with specific examples.

[0033] Step 1: Collect and organize historically accumulated project development documents and software source code files, project test documents, software test document templates, expert review opinions, national / industry standards, typical defects and other data assets, classify, screen and standardize them according to the corresponding standards, and build a high-quality hierarchical multimodal professional domain knowledge base; The project development documents include software requirements specifications, software design specifications, overall development requirements, technical solutions, and other project documents provided by the software development unit.

[0034] The project test documentation includes software test outlines, software test instructions, software test records, software test reports, and other software test process documents.

[0035] The software test document templates include software test outline templates, software test instruction templates, software test record templates, software test report templates, and other document templates generated during the software testing process.

[0036] The software testing outline includes: test object, test environment, name of configuration item of the software under test, test type, test item name, test item identifier, relationship with development documentation, test sub-items and their descriptions, etc.

[0037] The software test description includes: an overview of the software under test (including software name, software type, programming language, development environment, operating environment, development unit, etc.), test environment, configuration item names of the software under test, test type, test case name, test case identifier, relationship with the test outline, test description, test case initialization, prerequisites and constraints, termination conditions, and test process (including detailed test steps: input and operation instructions, expected test results, and evaluation criteria).

[0038] The high-quality, hierarchical, multimodal professional domain knowledge base includes a test case text library, a test case association graph library, a software code library, a test document template library, an expert review knowledge base, a national / industry standard library, and a typical defect library.

[0039] The test case text library is formed by hierarchically classifying, filtering, and standardizing project data (project development documents, project test documents) closely related to test cases in data assets. This data is then further segmented (chunking), embedded (embedding), vectorized, and indexed to create the test case text library. The test case text library employs a hybrid retrieval strategy, combining semantic search (calculating the cosine similarity between the query vector and the chunk vector) and keyword search. The results from both are weighted and fused (e.g., 0.7 × semantic score + 0.3 × keyword score) before reranking, recalling the top K most relevant results (e.g., Top 5). Reranking uses a cross-encoder model (e.g., bge-reranker-large) to refine the initial retrieval results, recalling the top K most relevant chunks and improving the relevance of the Top N results. The hierarchical classification, screening, and standardization process involves three levels of classification: first, by the type of software being tested (including embedded software, non-embedded software, and FPGA software); second, by the professional field of the tested objects (e.g., satellite control systems, aircraft inertial navigation); and third, by the type of test (e.g., functional testing, performance testing, interface testing, and security testing). This process ensures high-quality data screening. Simultaneously, a unique identifier ID system is constructed using a hierarchical combination coding method to generate unique identifier IDs for data assets, facilitating bidirectional tracking of requirements, code, and use cases. Specifically, text is uniformly formatted as Markdown. Tables in documents are converted to Markdown format, and titles and explanatory text are added before in-situ replacement. Images (flowcharts, sequence diagrams, etc.) in documents are identified using tools combined with OCR, and key information is extracted and converted into corresponding text descriptions before in-situ replacement. Formulas in documents are identified, key information is extracted, and converted into LaTeX code format before in-situ replacement.

[0040] A unique identifier ID system is constructed using a hierarchical combination coding method.

[0041] (1) The asset type (identified object) is defined as follows: Document assets (document type abbreviations) include: SRS (Software Requirements Specification); SDS (Software Design Specification); TSL (Technical Solution); TSD (Test Specification Document); TDD (Test Description Document); and TRP (Test Report).

[0042] The code assets include: SRC source code; CLA class; FUC function; and CDS code snippet.

[0043] The knowledge units include: DEP (Document Extracted Paragraph); TBL (Table); PIC (Picture); TRP (Test Requirement Point); SCI (Software Configuration Item); FNT (Functional Test); PFT (Performance Test); INT (Interface Test); SCT (Security Test); TST (Test Item); TSI (Test Subitem); and STC (Software Test Case).

[0044] (2) ID Coding Standard Specification: A hierarchical combination coding method is uniformly adopted to ensure the uniqueness, readability, and resolvability of IDs. Among them, the hierarchical sequence number of software configuration items, test items, test sub-items, and test cases can reflect their position in the test tree.

[0045] For example, the ID encoding standard specification can be specifically defined as: ① Project ID (Project_ID): PRJ{Software type number (2 digits)}{Professional field number (2 digits)}{Project year and month YYYYMM (6 digits)}{Serial number (3 digits)}.

[0046] The software type numbering rules are as follows: Embedded software: 01; Non-embedded software: 02; FPGA software: 03.

[0047] Among them, the professional field number is a unified professional field number formulated by this unit for different professional fields. For example, the professional field number for satellite control system is 01, and the professional field number for aircraft inertial navigation is 02, etc.

[0048] The serial number is a unique sequence number within the same level.

[0049] For example, the embedded software test project ID for the 16th aircraft inertial navigation specialty in June 2025 is PRJ0102202506016.

[0050] ②Document ID (Doc_ID): {Project_ID}-{Document type abbreviation}-{Version number}.

[0051] For example, the test outline document ID for the 16th embedded software test project of the aircraft inertial navigation major in June 2025 is: PRJ0102202506016-TSD-V1.2.

[0052] ③ Software configuration item ID (SCI_ID): {Doc_ID}-SCI{serial number (2 digits)}.

[0053] For example, the second software configuration item ID in the test outline document of the 16th embedded software test project for aircraft inertial navigation in June 2025 is: PRJ0102202506016-TSD-V1.2-SCI02.

[0054] ④ Test Item ID (TestItem_ID): {SCI_ID}-{Test Type (FNT / PFT / INT / SCT, etc.)}-TST{Hierarchical Sequence Number (3 digits)}.

[0055] For example, the fifth test item ID of the functional test of the second software configuration item in the test outline document of the 16th embedded software test project of the aircraft inertial navigation major in June 2025 is: PRJ0102202506016-TSD-V1.2-SCI02-FNT-TST005.

[0056] ⑤ Test Subitem ID: {TestItem_ID}-TSI{hierarchical serial number (3 digits)}.

[0057] For example, in the test outline document of the 16th embedded software test project for aircraft inertial navigation in June 2025, the second test sub-item ID of the 5th test item of the second software configuration item is: PRJ0102202506016-TSD-V1.2-SCI02-FNT-TST005-TSI002.

[0058] ⑥ Test Case ID (TestCase_ID): {TestSubitem_ID}-STC{hierarchical sequence number (3 digits)}.

[0059] For example, the first test case ID of the second test sub-item of the fifth test item in the functional test of the second software configuration item in the test outline document of the 16th embedded software test project for aircraft inertial navigation in June 2025 is: PRJ0102202506016-TSD-V1.2-SCI02-FNT-TST005-TSI002-STC001.

[0060] ⑦ Code File ID (File_ID): {Project_ID}-SRC-{hash value (8 bits) of the relative path of the code file in the repository}, for example: PRJ2024100101-SRC-a1b2c3d4.

[0061] ⑧ Class ID: {File_ID}-CLA-{Class Name}-{Start Line Number}_{End Line Number}.

[0062] ⑨ Function ID: {File_ID}-FUN-{function name}-{start line number}_{end line number}.

[0063] ⑩ Code Snippet ID: {File_ID}-CDS-{start line number}_{end line number}.

[0064] ⑪ Test Requirement Point ID (TRP_ID): {Doc_ID}-{Chapter Number}-TRP{Serial Number (3 digits)}.

[0065] For example, the first requirement point ID in section 3.2.2 of the requirements specification document for the 16th embedded software testing project for aircraft inertial navigation in June 2025 is: PRJ0102202506016-SRS-V1.2-3.2.2-TRP001.

[0066] The test case association graph library is a hierarchical graph database that uses a tree diagram to represent the hierarchical relationships between configuration items, test types, test items, test sub-items, test cases, source code snippets / functions, and test requirements of the software under test. This effectively reveals the multi-level relationships and tracing between "test requirements - test items - test cases - software code." The test case association graph library uses a graph database (such as Neo4j) to store all nodes and relationships and provides query language interfaces such as Cypher to support complex searches. Its core function is to visualize relationships and display structured hierarchies graphically, thereby assisting testers in quickly designing test cases, ensuring complete test case coverage, and improving test case design efficiency.

[0067] The software code library is initialized according to a unified standard format. The software code files and the functions they contain are labeled with metadata and stored in the library according to the software type, professional field, project name and identifier ID of the project, thus forming the software code library. The software code library employs structured storage (source code, documentation, metadata, etc. are stored separately; source code is parsed and stored by file, function, class, and code block). It supports keyword and semantic similarity retrieval after metadata slicing and vectorization. It supports source code parsing for multiple languages ​​(for example, the tree-sitter tool can be used, which supports syntax parsing for multiple languages ​​such as Python, Java, C, C++, Go, and JavaScript, providing syntax definitions for each language and generating a syntax tree (AST). The specific process is: traversing the project source code -> parsing each file using tree-sitter -> traversing the AST -> identifying function definitions, class definitions, and other nodes -> extracting their names, parameters, start line numbers, end line numbers, comments, etc. -> storing them in the structured code library). It supports ID retrieval (each code file, class, function, and code snippet has a unique ID), supports text search (keyword search for function names, variable names, comments, etc.), and provides API linkage interfaces for the test case text library and test case association graph library to call, realizing cross-library linkage between multiple libraries.

[0068] The test document template library provides standardized test document templates for various professional fields, which can be retrieved and reused when generating various software testing process documents such as software test outline documents, software test instruction documents, software test record documents, and software test report documents.

[0069] The expert review knowledge base is formed by refining, summarizing, segmenting, embedding vectorization, and indexing the professional opinions, suggestions, and expert experience and knowledge put forward by experts in various professional fields during the review of test outlines, test cases, and other test projects.

[0070] The national / industry standard library is a collection of relevant national / industry standards and enterprise standard specifications that need to be followed during software testing. After being collected, processed, segmented, embedded, vectorized, and indexed, the national / industry standard library is formed, supporting accurate retrieval and citation of standard clauses.

[0071] The typical defect library is a collection of typical software defects accumulated during the execution of software testing projects. It is categorized, labeled, segmented, embedded, vectorized, and indexed according to three levels: first-level classification (including embedded software, non-embedded software, and FPGA software), second-level classification (e.g., satellite control system, aircraft inertial navigation, etc.), and third-level classification (e.g., functional testing, performance testing, interface testing, security testing, etc.). This library can provide references for negative test scenarios and boundary condition design for test case design, and helps to improve the defect detection rate.

[0072] Step 2: The user submits a task to generate high-quality test cases based on the software requirements specification document, software design specification, overall development requirements, technical solutions and other development documents and software source code of the project to be tested through the human-computer interaction interface. The task planning agent identifies the user's intent and coordinates the planning, and then distributes the task to the test requirements analysis agent.

[0073] The human-computer interface (HCI) serves as the window for interaction between the user and the system, effectively transmitting user input to the system and displaying system output to the user in various formats. The HCI displays the test tree diagram generated from the large model and allows testers to mark whether it meets requirements. It also supports adding, deleting, and modifying nodes in the diagram, and allows testers to provide feedback on the generated content. For generated content that does not meet requirements, it can be manually edited directly in the HCI or regenerated by the large model as needed, until all relevant content meets the requirements. Specifically, testers can adjust the input prompts for the large model through the HCI, enabling it to better complete tasks such as test tree diagram generation, test outline generation, and test case generation based on the tester's latest intentions.

[0074] The task planning agent can identify the input user intent, then coordinate and plan the task implementation path and select an appropriate workflow, and distribute the task to relevant agents for execution. Specifically, in the software testing process, users may propose tasks such as generating test outlines based on input requirements and code, generating test cases based on input requirements and code, or generating test cases based on input test outlines. The task planning agent can coordinate and plan the task implementation path and required agents for different task requirements, and can efficiently complete specific tasks by scheduling multiple agents to collaborate. It adopts a collaborative design using the MCP (Model Context Protocol) and A2A (Agent-to-Agent) protocols to achieve efficient communication between agents and tools, data sources, and among agents themselves.

[0075] Step 3: The test requirement analysis agent performs preprocessing on the input development documents and software source code files of the project under test, such as software requirement specifications, software design specifications, overall development requirements, and technical solutions. These processes include classification, segmentation, parsing, splitting, code comment completion, metadata injection and annotation, and error review and correction. Then, the dynamic prompt word construction engine dynamically assembles new complete prompt word input models based on the current test requirement analysis mining tasks. This guides the large model to further extract and annotate the test requirement points, such as functions, performance, interfaces, and security, involved in each configuration item of the software under test, along with their descriptions (unique identifiers, and corresponding tracking relationships of code snippets or functions in the software source code, etc.). This constructs a preliminary test requirement-code snippet / function set. After manual review, confirmation, optimization, and completion through a human-computer interaction interface, the {test requirement-code snippet / function set} is formed.

[0076] The Large Language Model (LLM) is a model in the field of artificial intelligence (AI) built using deep learning technology, focused on processing and generating human language. Its core characteristic is that, through training on large-scale text data, it possesses the ability to understand language semantics and grammatical rules, and generate text content that conforms to human expression habits.

[0077] Step 4: The test tree diagram generation agent analyzes the {test requirements - code snippets / function sets} provided by the agent and simultaneously searches multimodal professional domain knowledge bases (test case text library, test case association graph library, software code library, typical defect library, etc.). The search results are weighted and integrated and then input into the dynamic prompt word construction engine. The engine dynamically assembles new complete prompt words based on the current test tree diagram generation task and inputs them into the large model to generate the corresponding test tree diagram (including software configuration items, test types (functional tests, performance tests, interface tests, etc.), test items, test sub-items, and labeled with unique identifiers ID). The tree diagram is then displayed to the testers through a human-computer interaction interface. After manual review, confirmation, optimization, and completion, the first test tree diagram is generated.

[0078] The dynamic prompt word construction engine can automatically select different roles and modules according to different task requirements, and dynamically assemble the various components into a complete large model prompt word in MARKDOWN format adapted to the current task, according to a standardized prompt word template architecture. Specific steps include: 1) Select the main role, auxiliary role, and corresponding ability / behavioral guidelines required to complete the task from the system role library according to the task requirements, and insert the prompt word template into the corresponding position.

[0079] The system role library is a database of various role definitions and their corresponding abilities / behavioral guidelines built and maintained in the system. Each role is a "domain expert" with specific skills and perspectives.

[0080] For example: ① Role: Test Requirements Analyst.

[0081] Corresponding task requirements: Test requirements analysis and related task discovery.

[0082] Competencies / Code of Conduct: Proficient in interpreting and analyzing development documents such as software requirements specifications, software design specifications, overall development requirements, and technical solutions; able to accurately identify functional requirements, non-functional requirements (performance, interfaces, security), business rules, input / output constraints, potential business states and transitions. Output structured, unambiguous requirements and their descriptions.

[0083] ② Role: Test Case Designer.

[0084] Corresponding task requirements: Test case generation related tasks.

[0085] Competencies / Behavioral Guidelines: Proficient in writing high-quality, executable test cases; skilled in various test case design methods (equivalence partitioning, boundary value analysis, decision tables, cause-effect graphing, etc.); able to design detailed test steps, test data, preconditions, and expected results.

[0086] ③ Role: Code Analysis Expert.

[0087] Corresponding task requirements: code analysis, code comment completion / correction, and other code-related tasks.

[0088] Competencies / Code of Conduct: Proficient in static code analysis, able to understand code logic, identify key functions, branch conditions, exception handling logic, API calls, and potentially risky code segments; able to complete / correct file-level and function-level code comments.

[0089] 2) Insert the task input into the corresponding position in the prompt word template.

[0090] 3) Insert the contextual knowledge into the corresponding position of the prompt word template.

[0091] 4) Task Thinking and Execution Planning. This section requires intent recognition and thinking for specific input tasks, breaking down the tasks into various sub-tasks in the business workflow, and selecting the appropriate prompt word sub-modules for execution for different sub-tasks.

[0092] The prompt submodule is a module (including a task output requirement module) that dynamically and adaptively selects and assembles prompt words for different task execution plans. Each prompt submodule predefines the steps for breaking down tasks / subtasks and reasoning step by step for different tasks / subtasks in a chain-of-thought (COT) and tree-of-thought (TOT) manner, including various tools that may be called during the process (e.g., to obtain the corresponding computing resource management tools based on the current computing resource usage of the system), and provides detailed reasoning requirements and process descriptions for intermediate steps.

[0093] 5) Association and Traceability: This section associates the generated content with the requirement ID, code ID, etc. provided in the "Background Knowledge" section to ensure effective tracing.

[0094] 6) Review and Optimization: Review whether the generated content meets the input task requirements, whether it fully considers various scenarios in conjunction with background knowledge, whether it uses professional and unambiguous test terminology, and optimize and improve any content that does not meet the requirements.

[0095] 7) Output Requirements: For different task inputs, select the appropriate output prompt submodule and insert it into the corresponding position of the prompt template. For different tasks, specify the content to be output / not output, the output format (e.g., JOSN format), identify it with a unified ID, and prohibit the output of irrelevant content.

[0096] 8) Specific Examples: List one or two typical examples of the corresponding task.

[0097] For example, the prompt word template structure is as follows: { ## You are a senior {character}, {ability / behavioral guidelines} ## Task Input: {Task _Input} ## Contextual Knowledge ## Task Thinking and Execution Planning: Please follow the steps below to think about and execute the plan, and finally complete {Task _Input}.

[0098] ### 1.{Prompt Submodule}......; ### 2.{Prompt Submodule}......; ### 3.{Prompt Submodule}......; ## Association and Tracing: - Requirement association: Requirement ID: xxxx - Code snippet / function association: Code ID: xxxx ## Review and Optimization: Please review whether the generated content meets the input task requirements, whether it fully considers various scenarios in conjunction with contextual background knowledge (such as consideration of normal and abnormal, boundary, etc. in test case generation tasks), whether it uses professional and unambiguous test terminology, and optimize and improve any non-compliant content.

[0099] ## Output Requirements: {Output Prompt Submodule} ## Specific Example: {Specific Example} } The first test tree diagram is a hierarchical tree structure diagram, which includes test items, test sub-items, test types, configuration items of the software under test, and the tracing relationship between test items and test requirement points, code snippets / functions.

[0100] Step 5: The test outline generation agent performs task analysis and evaluation based on the first test tree diagram provided by the test tree diagram generation agent. It dynamically plans the number of tasks that can be generated in parallel using multiple threads based on the number of test items and sub-items, as well as the current hardware computing resources of the system. When executing specific generation tasks, it searches a multimodal professional domain knowledge base in parallel, weights and fuses the search results, and inputs them into a dynamic prompt word construction engine. This engine dynamically assembles new complete prompt words based on the current test outline generation task, inputting them into a large model to further generate descriptions of each test item, each test sub-item, and corresponding detailed test method descriptions for each test type. Each generated test sub-item... The descriptions and other content are all manually reviewed, optimized, supplemented, and confirmed (in particular, for content generated by large models that does not meet the requirements, manual editing and modification can be performed directly in the human-computer interaction interface, or, depending on the situation, the large model can be regenerated and then manually selected and reviewed). All of this content is then input into the test document to automatically generate an intelligent agent, which calls the test outline template of the corresponding professional field to generate a test outline document that meets the template requirements. The test outline document is then input into the review intelligent agent for review, and the review opinions are put forward. After manual confirmation, it is sent to the test outline generation intelligent agent to modify the issues raised to close the loop. Finally, after manual review and confirmation, the final version of the test outline document is output.

[0101] The dynamic planning of the number of tasks that can be generated in parallel by multiple threads based on the number of test items and test sub-items and the current hardware computing resources of the system is to improve efficiency and maximize the use of the current hardware computing resources of the system. The intelligent agent generated by the test outline first evaluates the number of tasks that can be generated in parallel by multiple threads based on the current task volume and the current system computing resources (which can be obtained by calling tools). Then, based on the evaluation results, the tasks are divided into batches and threads, and then the specific generation tasks are executed in parallel by multiple threads.

[0102] The test document automatic generation agent can autonomously select a test document template that matches the specific professional field from the test document template library according to the different types of test document generation task requirements (such as test outline generation, test instruction generation, etc.), and fill the input content into the corresponding chapters of the test document template, and finally output a complete test document.

[0103] The review agent can retrieve multimodal professional domain knowledge bases (especially expert review knowledge bases (the search results have higher weight than other knowledge bases) and national / industry standard bases) according to different review tasks (such as test outline review, test instruction review, etc.), and input the weighted fusion of the search results into the dynamic prompt word construction engine, which dynamically assembles new complete prompt words according to the current review task and inputs them into the large model to generate specific review opinions.

[0104] Step 6: The test tree diagram generation agent generates the final version of the test outline based on the first test tree diagram and the test outline. It searches the multimodal professional domain knowledge base in parallel, and inputs the weighted and fused search results into the dynamic prompt word construction engine. The engine dynamically assembles new complete prompt words into the large model based on the current test tree diagram generation task to further generate a complete test tree diagram (including each test case node corresponding to each test sub-item and its unique identifier ID, etc.). The tree diagram is then displayed to the testers through a human-computer interaction interface. After manual review, confirmation, optimization and completion, the second test tree diagram is generated.

[0105] Step 7: The test case generation agent performs task analysis and evaluation on the final version of the test outline provided by the test outline generation agent and the second test tree diagram provided by the test tree diagram generation agent. Based on the number of test sub-items and the current hardware computing resources of the system, it dynamically plans the number of tasks that can be generated in parallel by multiple threads. When executing specific generation tasks, it searches the multimodal professional domain knowledge base in parallel, and inputs the weighted fusion of the search results into the dynamic prompt word construction engine. The engine then dynamically assembles new complete prompt words into the large model based on the current test case generation task. This model further generates all test cases corresponding to each test sub-item of each test type (including unique identifier ID, test description, test case initialization, prerequisites and constraints, termination conditions, test process (specifically including: input and operation instructions, expected test results, evaluation criteria, etc.), as well as the tracking relationship between each test case and the test outline, the tracking relationship between the test requirement points, and the corresponding tracking relationship between the test case and the software code of this project. Specifically, test cases generated by the large model that do not meet the requirements can be edited directly by humans through the human-computer interaction interface, or, depending on the situation, regenerated by the large model and then manually selected and reviewed. All generated test cases are scored and judged by humans (subjectively evaluating the generated test cases based on the testers' experience in terms of correctness, relevance, standardization, logical rationality, scenario applicability, and contextual consistency), and objectively evaluating the test case quality and reflecting on the intelligent agent (laying the foundation for subsequent collection of human feedback data for reinforcement learning-based model optimization), and then reviewed by testers. After verification, a complete test case set is formed. Based on the complete test case set, the test tree diagram generation agent is called to generate a second test tree diagram that updates the second test tree diagram. This updated test tree diagram is then displayed to the testers through a human-computer interaction interface. The second test tree diagram and the complete test case set are then input into the test document automatic generation agent, which calls the test case document template of the corresponding professional field to generate a test specification document. The test specification document is then input into the review agent for review, which provides review comments on the test specification document (especially the test cases). After manual confirmation, the document is sent back to the test case generation agent to modify and close the loop, and then the final version of the test specification document is output.

[0106] The test case quality assessment and reflection agent can automatically invoke relevant tools to quantitatively evaluate the quality of generated test cases from multiple dimensions, including test case adoption rate, requirement coverage, and executability. Specifically, this includes: 1) Calculate the test case adoption rate: The adoption rate is the ratio of test cases that are directly adopted or adopted with slight modifications to the total number of generated test cases. The formula is as follows: ; Where N1 represents the number of test cases directly adopted, N2 represents the number of test cases adopted with slight modifications, and N represents the total number of test cases adopted.

[0107] 2) Calculate demand coverage: Requirements coverage is calculated using a comparative analysis of traceability relationships, revealing the ratio of the number of test requirements covered by generated test cases to the total number of test requirements. The formula is as follows: ; Where R1 represents the number of test requirement points that the generated test cases can trace and cover, and R represents the total number of test requirement points.

[0108] 3) Computational executability: Executability is the proportion of generated test cases that are directly executable out of the total number of test cases. The formula for calculation is: ; Where M represents the number of test cases generated that are directly executable, and N represents the total number of test cases.

[0109] 4) Comprehensive evaluation: The evaluation results of three dimensions—test case adoption rate, requirement coverage, and executability—are weighted and integrated to achieve a comprehensive evaluation of test case quality. The specific calculation formula is as follows: ; Where w1 represents the weight of the trial adoption rate metric, w2 represents the weight of the requirement coverage metric, and w3 represents the weight of the feasibility metric. In practice, w1 can take a value of 0.3, w2 can take a value of 0.4, and w3 can take a value of 0.3.

[0110] Meanwhile, the test case quality assessment and reflection agent can generate the final version of the test specification document provided by the agent based on the test cases. It combines the development documents and software source code of the task planning agent input by the user, as well as the subjective and objective evaluation results of the test case quality. By parallel searching of the multimodal professional domain knowledge base and weightedly fusing the search results, it inputs them into the dynamic prompt word construction engine. The engine then dynamically assembles a new complete prompt word input model based on the current test case quality assessment and reflection task to evaluate and reflect on the overall task completion. It proposes improvement suggestions for shortcomings, and if necessary, it can input the suggestions into the task planning agent to re-plan the task and iteratively optimize it until the task is completed with high quality (as determined by the user).

[0111] Step 8: Upon project completion, the knowledge assets generated during the project execution (input requirement specifications, software source code files, and generated test requirement sets, test tree diagrams, test items, test sub-items, test cases, test process descriptions, test outlines, test documentation, etc.) are categorized, screened, standardized with metadata, and reviewed by the knowledge management agent. After manual review and confirmation, they are incorporated into the corresponding high-quality, multimodal, professional domain knowledge base, forming knowledge accumulation that is evolvable and progressive. Simultaneously, statistical analysis is performed on the adoption rate and requirement coverage of test cases generated by the agent, and the recall status of each knowledge block in the knowledge base is statistically analyzed. Knowledge blocks that have not been recalled for a long time are periodically analyzed for the reasons and optimized to ensure their effectiveness. The entire system possesses self-learning and continuous evolution capabilities, and the quality of generated test cases will continuously improve as the knowledge base becomes richer.

[0112] refer to Figure 5 Another specific embodiment of the present invention discloses a test case generation system based on efficient multi-agent collaboration, comprising: a multimodal professional domain knowledge base construction module 501, used to hierarchically classify, filter, and standardize project development documents, software source code files, project test documents, software test document templates, expert review opinions, national / industry standards, and typical defects of historical projects to construct a multimodal professional domain knowledge base; a task planning agent module 502, used to generate test case generation tasks based on the project development documents and software source code files of the project to be tested and distribute them to the test requirements analysis agent; a test requirements analysis agent module 503, used to extract test requirement points and their descriptions involved in the configuration items of the software under test based on the test case generation tasks and mark the correspondence between the test requirement point ID and the code snippets or functions in the software source code files to construct a test requirement-code snippet / function set; and a test case generation agent module 506, used to generate test cases and test specification documents based on the test requirement-code snippet / function set and the multimodal professional domain knowledge base.

[0113] In the following text, refer to Figure 4 The present invention provides a detailed description of a test case generation system based on efficient multi-agent collaboration according to embodiments of the present invention, using specific examples.

[0114] refer to Figure 4 and Figure 5This system organically combines the Multi-Agent and ReAct (Reasoning and Acting) models, employing the MCP (Model Context Protocol) and A2A (Agent-to-Agent) protocols for collaborative design to achieve efficient communication between agents and tools / data sources, as well as among agents themselves. The system comprises both software and hardware. The software mainly includes: a multimodal professional domain knowledge base construction module 501, a task planning agent module 502, a test requirements analysis agent module 503, a test tree diagram generation agent module 504, a test outline generation agent module 505, a test case generation agent module 506, a test case quality assessment and reflection agent module 507, a knowledge management agent module 508, a human-computer interaction interface module 509, a dynamic prompt word construction engine module 510, a test document automatic generation agent module 511, and a review agent module 512. The hardware mainly includes a server and a client.

[0115] The multimodal professional domain knowledge base construction module 501 is used to collect and organize historically accumulated project development documents and software source code files, project test documents, software test document templates, expert review opinions, national / industry standards, typical defects, and other data assets. These data assets are then categorized, filtered, and preprocessed according to corresponding standards to construct a high-quality, hierarchical multimodal professional domain knowledge base. This multimodal professional domain knowledge base includes a test case text library, a test case association graph library, a software code library, a test document template library, an expert review knowledge base, a national / industry standard library, and a typical defect library.

[0116] The task planning agent module 502 can identify the input user intent, then coordinate and plan the task implementation path, and distribute the task to the relevant agent modules for execution. Specifically, in the software testing process, users may propose tasks such as generating test outlines based on input requirements and code, generating test cases based on input requirements and code, or generating test cases based on input test outlines. The task planning agent module can coordinate and plan the task implementation path and required agent modules for different task requirements, and can efficiently complete specific tasks by scheduling multiple agent modules to collaborate.

[0117] The test requirements analysis intelligent agent module 503 performs preprocessing on the input development documents and software source code files of the project under test, such as software requirements specifications, software design specifications, overall development requirements, and technical solutions. These preprocessing includes classification, segmentation, parsing, splitting, code comment completion, metadata injection and annotation, and error review and correction. Then, it uses a large model to extract the test requirements points, such as functions, performance, interfaces, and security, involved in the configuration items of each software under test, and their descriptions and annotations (unique identifiers, and corresponding tracking relationships of code snippets or functions in the software source code, etc.). This constructs a preliminary test requirements-code snippet / function set, which is then manually reviewed, confirmed, and optimized through the human-computer interaction interface module to form the {test requirements-code snippet / function set}.

[0118] The dynamic prompt word construction engine module 510 is used to dynamically assemble various parts to build large model prompt words according to different task requirements. The prompt word template components include roles and abilities / behavioral guidelines, task input, contextual background knowledge, task thinking and execution planning, association and tracing, review and optimization, output requirements, specific examples, etc.

[0119] The test tree diagram generation agent module 504 generates the final test outline provided by the agent module based on the {test requirements - code snippets / function sets} or the first test tree diagram and the test outline provided by the test requirements analysis agent module. At the same time, it searches the multimodal professional domain knowledge base (test case text library, test case association graph library, software code library, typical defect library, etc.) in parallel, and inputs the weighted fusion of the search results into the dynamic prompt word construction engine module 510. The engine module dynamically assembles new complete prompt words into the large model based on the current test tree diagram generation task to generate the corresponding test tree diagram (including software configuration items, test types (functional tests, performance tests, interface tests, etc.), test items, test sub-items, and marked with a unique identifier ID). The test tree diagram is then displayed to the testers through the human-computer interaction interface module. After manual review, confirmation, optimization and completion, the corresponding test tree diagram is generated.

[0120] The test outline generation agent module 505 performs task analysis and evaluation based on the first test tree diagram provided by the test tree diagram generation agent module. It dynamically plans the number of multi-threaded parallel batch generation tasks based on the number of test items and sub-items, as well as the current system hardware computing resources. During the execution of specific generation tasks, it searches a multimodal professional domain knowledge base in parallel, weights and fuses the search results, and inputs them into a dynamic prompt word construction engine. This engine dynamically assembles new complete prompt words based on the current test outline generation tasks, inputting them into a large model to further generate descriptions of each test item, each test sub-item, and corresponding detailed test method descriptions for each test type. Each generated... The descriptions of each test sub-item and other content are all manually reviewed, optimized, supplemented, and confirmed (in particular, for generated content that does not meet the requirements, it can be generated multiple times by the large model and then manually selected and reviewed). All of this content is then input into the test document automatic generation intelligent agent module 511, which calls the test outline template of the corresponding professional field to generate a test outline document that meets the template requirements. The test outline document is then input into the review intelligent agent module 512 for review, and the review opinions are put forward. After manual confirmation, it is sent to the test outline generation intelligent agent module to modify and close the loop of the issues raised. Finally, after manual review and confirmation, the final version of the test outline document is output.

[0121] The test document automatic generation intelligent agent module 511 can automatically select a test document template that matches the specific professional field from the test document template library according to the different types of test document generation task requirements (such as test outline generation, test instruction generation, etc.), and fill the input content into the corresponding chapters of the test document template, and finally output a complete test document.

[0122] The review agent module 512 can retrieve multimodal professional domain knowledge bases (especially expert review knowledge bases (the retrieval results have higher weight than other knowledge bases) and national / industry standard bases) according to different review tasks (such as test outline review, test instruction review, etc.), and input the weighted fusion of the retrieval results into the dynamic prompt word construction engine, which dynamically assembles new complete prompt words according to the current review task to generate specific review opinions from the large model.

[0123] The test case generation agent module 506 performs task analysis and evaluation on the final test outline provided by the test outline generation agent module and the second test tree diagram provided by the test tree diagram generation agent module. Based on the number of test sub-items and the current hardware computing resources of the system, it dynamically plans the number of multi-threaded parallel batch generation tasks. When executing specific generation tasks, it searches a multimodal professional domain knowledge base in parallel, weights and fuses the search results, and inputs them into a dynamic prompt word construction engine. This engine dynamically assembles new complete prompt words into a large model based on the current test case generation task. The large model further generates all test cases corresponding to each test sub-item of each test type (including unique identifier ID, test description, test case initialization, prerequisites and constraints, termination conditions, test process (specifically including: input and operation instructions, expected test results, evaluation criteria, etc.)), as well as the tracking relationships between each test case and the test outline, the test requirement points, and the corresponding software code of this project. All generated test cases are manually scored and evaluated (based on the test outline's performance). The testers' experience allows for subjective evaluation of the generated test cases (in terms of correctness, relevance, standardization, logical rationality, scenario applicability, and contextual consistency) and objective evaluation by the test case quality assessment and reflection intelligent agent module (laying the foundation for subsequent model optimization based on reinforcement learning by collecting human feedback data). After review and confirmation by the testers, a complete test case set is formed. Based on the complete test case set, the test tree diagram generation intelligent agent module is called to generate a third test tree diagram after updating the second test tree diagram. This third test tree diagram is then displayed to the testers through a human-computer interaction interface. The third test tree diagram and the complete test case set are then sent to the test document automatic generation intelligent agent module 511, which calls the test case document template of the corresponding professional field to generate a test specification document. The test specification document is then input into the review intelligent agent module 512 for review, which provides review opinions on the test specification document (especially the test cases). After manual confirmation, the document is sent back to the test case generation intelligent agent module for modification and closure of the loop, and then the final version of the test specification document is output.

[0124] The Test Case Quality Assessment and Reflection Agent Module 507 can automatically invoke tools to quantitatively assess the quality of generated test cases from multiple dimensions, including adoption rate, requirement coverage, and executability. It can generate the final version of the test specification document provided by the agent module based on the test cases. Combining the user-input task planning agent module's development documentation and software source code, as well as subjective and objective evaluations of test case quality, it performs parallel searches of a multimodal professional domain knowledge base. The weighted and fused search results are then input into a dynamic prompt word construction engine, which dynamically assembles new, complete prompt words based on the current test case quality assessment and reflection task. This engine then evaluates and reflects on the overall task completion, proposing improvements for shortcomings. If necessary, it can re-input the prompts into the task planning agent module for re-planning and iterative optimization until the task is completed with high quality (as determined by the user).

[0125] The Knowledge Management Intelligent Agent Module 508 categorizes, filters, standardizes metadata, and reviews the knowledge assets generated during the project execution process (input requirement specifications, software source code files, and generated test requirement sets, test tree diagrams, test items, test sub-items, test cases, test process descriptions, test outlines, test documentation, etc.). After manual review and confirmation, these assets are incorporated into a high-quality, multimodal, professional domain knowledge base of the corresponding type, forming knowledge accumulation that is evolvable and progressive. Simultaneously, the module statistically analyzes the adoption rate and requirement coverage of test cases generated by the intelligent agent module, and statistically analyzes the recall status of each knowledge block in the knowledge base. Knowledge blocks that have not been recalled for a long time are periodically analyzed for reasons and optimized to ensure their effectiveness. The entire system possesses self-learning and continuous evolution capabilities, and the quality of generated test cases will continuously improve as the knowledge base becomes richer.

[0126] The human-computer interface module 509 serves as the window for user-system interaction, effectively transmitting user input to the system and displaying system output to the user in various formats. This module displays the test tree diagram generated by the test tree diagram generation agent, allowing testers to mark whether it meets requirements, add, delete, and modify nodes, and provide feedback on the generated content. For generated content that does not meet requirements, manual editing can be performed directly in the human-computer interface, or the large model can be regenerated as needed, until all content meets the requirements. Specifically, testers can adjust the input prompts for the large model through the human-computer interface, enabling it to better complete tasks such as test tree diagram generation, test outline generation, and test case generation based on the tester's latest intentions.

[0127] The server-side includes computing resources such as CPUs and GPUs, hardware storage resources such as memory and disk arrays, software resources such as operating systems, middleware, and databases, and network resources such as firewalls, switches, and routers. The test case generation system software adopts a B / S architecture, and both it and the multimodal professional domain knowledge base resources are deployed on a remote server. Furthermore, considering data security, a large model and its inference framework are also privately deployed on the server. Testers remotely access the system's human-computer interaction interface through a browser on the internal LAN, and the data generated during the interaction is stored in the database on the server.

[0128] The client has a browser installed, which is the user terminal for testers to access the test case generation system on the remote server and achieve effective interaction through the human-computer interaction interface.

[0129] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0130] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A test case generation method based on efficient cooperation of multiple agents, characterized in that, The method comprises the following steps: Hierarchical classification and standardized processing of project development documents, software source code files, project test documents, software test document templates, expert review opinions, national / industry standards, and typical defects of historical projects are performed to construct a multi-modal professional domain knowledge base; A test case generation task is generated based on the project development documents and software source code files of the to-be-tested project by a task planning agent module, and is distributed to a test requirement analysis agent; Based on the test case generation task, test requirement points involved in the software configuration item under test and their descriptions are extracted, and a corresponding tracking relationship between the test requirement point ID and the code snippet or function in the software source code file is labeled to construct a test requirement-code snippet / function set; Based on the test requirement-code snippet / function set and the multi-modal professional domain knowledge base, test cases and test specification documents are generated.

2. The test case generation method based on multi-agent efficient collaboration according to claim 1, characterized in that, After generating the test cases and test specification documents, further comprising: for the knowledge assets formed in the execution process of the test case generation task, automatic classification, screening, metadata standardization, and review are performed by a knowledge management agent, and then stored in the multi-modal professional domain knowledge base after manual review and confirmation through a human-computer interaction interface, wherein the knowledge assets include the project development documents, software source code files, test requirement-code snippet / function set, test cases, and test specification documents of the to-be-tested project.

3. The test case generation method based on multi-agent efficient collaboration according to claim 1, characterized in that, The hierarchical classification and standardized processing further comprises: First-level classification according to the type of the software under test involved in the project, wherein the first-level classification includes embedded software, non-embedded software, and FPGA software; Second-level classification according to the professional domain of the software under test involved in the project, wherein the second-level classification includes satellite control system, aircraft inertial navigation, and ship radar; Third-level classification according to the test type, wherein the third-level classification includes functional testing, performance testing, interface testing, and security testing; A unique identification ID system is constructed using a hierarchical combination coding method, and then a unique identification ID is generated for bidirectional tracking of requirements, code, and use cases.

4. The test case generation method based on multi-agent efficient collaboration according to claim 1, characterized in that, The task planning agent generates a test case generation task based on the project development documents and software source code files of the to-be-tested project and distributes it to the test requirement analysis agent further comprising: The project development documents and software source code files of the to-be-tested project are input into the task planning agent through a human-computer interaction interface, wherein the project development documents include software requirement specification documents, software design specifications, development general requirements, and technical solutions; The task planning agent identifies the test personnel's intention, then plans the task implementation path and selects the appropriate workflow, and distributes the test case generation task to the test requirement analysis agent; The task planning agent plans the implementation path of the task and the required agents according to different task requirements, so as to schedule multiple agents to efficiently cooperate to complete the test requirement analysis mining task, the test tree graph generation task, the test outline generation task, the test case generation task, the test case quality evaluation and reflection task, and the test document automatic generation task, and the knowledge management task.

5. The test case generation method based on multi-agent efficient cooperation according to claim 4, characterized in that, The MCP protocol and the A2A protocol are used to realize communication between different agents, between the agents and the multi-modal professional field knowledge base, and between the agents and the tools.

6. The test case generation method based on multi-agent efficient cooperation according to claim 5, characterized in that, The test case and the test description document are generated based on the test requirement-code fragment / function set and the multi-modal professional field knowledge base, and further include: A first-stage test tree graph is generated based on the test requirement-code fragment / function set; A final version of the test outline document is generated based on the first test tree graph; A second test tree graph is generated based on the first test tree graph and the final version of the test outline document; All test cases are generated based on the second test tree graph and the software test outline, and the quality of the all test cases is evaluated to generate a complete test case set; The test tree graph generation agent is called to update the second test tree graph according to the complete test case set; The complete test case set and the updated second test tree graph are input into the test document automatic generation agent, and the test case document template in the multi-modal professional field knowledge base is called to generate the test description document.

7. The multi-agent efficient cooperation-based test case generation method according to claim 6, wherein The first-stage test tree graph is generated based on the test requirement-code fragment / function set, and further includes: The test tree graph generation agent performs parallel retrieval on the multi-modal professional field knowledge base according to the test requirement-code fragment / function set, wherein the multi-modal professional field knowledge base includes a test case text library, a test case correlation graph library, a software code library and a typical defect library; The first retrieval result is weighted and fused and then input into a dynamic prompt word construction engine, and the dynamic prompt word construction engine dynamically assembles a first complete prompt word according to the test tree graph generation task of the to-be-tested project; The first complete prompt word is input into a large model to generate a corresponding test tree graph, and the test tree graph is audited, confirmed and optimized to generate a first test tree graph; The software test outline is generated based on the first test tree graph, and further includes: The test outline generation agent performs task analysis and evaluation on the first test tree graph, dynamically plans the number of multi-thread parallel batch generation tasks according to the number of test items and test sub-items and the current hardware computing resource situation, and performs parallel retrieval on the multi-modal professional field knowledge base; The second search result is input into the dynamic prompt word construction engine after weighted fusion, the dynamic prompt word construction engine generates a second complete prompt word according to a test outline of the to-be-tested project by dynamically assembling tasks; The second complete prompt word is input into a large model to generate each test item description, each test sub-item description, and a corresponding test method description corresponding to each test type; After optimizing and confirming the generated each test item description, each test sub-item description, and the corresponding test method description corresponding to each test type, the test document automatic generation agent is input; The test document automatic generation agent calls the test outline document template in the multi-modal professional field knowledge base to generate a required test outline document, and outputs a final version of the test outline document after modifying and confirming the test outline document.

8. The test case generation method based on multi-agent efficient collaboration according to claim 6, wherein, based on the first test tree graph and the final version of the test outline document, a second test tree graph is generated, further comprising: The test tree graph generation agent performs parallel retrieval on the multi-modal professional field knowledge base according to the first test tree graph and the final version of the test outline document; The third search result is input into the dynamic prompt word construction engine after weighted fusion, the dynamic prompt word construction engine generates a third complete prompt word according to the test tree graph of the to-be-tested project by dynamically assembling tasks; The third complete prompt word is input into a large model to generate a complete test tree graph, and a second test tree graph is generated after the complete test tree graph is confirmed and optimized; Based on the second test tree graph and the final version of the test outline document, a test case is generated, further comprising: The test case generation agent performs task analysis and evaluation on the final version of the test outline document and the second test tree graph, dynamically plans the number of multi-thread parallel batch generation tasks according to the number of test sub-items and the current hardware computing resource situation, and performs parallel retrieval on the multi-modal professional field knowledge base; The fourth search result is input into the dynamic prompt word construction engine after weighted fusion, the dynamic prompt word construction engine generates a fourth complete prompt word according to the test case generation task of the to-be-tested project by dynamically assembling tasks; The fourth complete prompt word is input into a large model to generate all test cases of each test sub-item corresponding to each test type. The quality of the all test cases is evaluated to generate a complete test case set, further comprising:

9. The test case generation method based on multi-agent efficient cooperation according to claim 8, characterized in that, The all test cases are displayed to the test personnel through a human-computer interaction interface, and the quality of the test cases is subjectively evaluated by scoring and judging through the test personnel, wherein the test cases generated by the large model that do not meet the requirements are directly edited, modified, and confirmed through the human-computer interaction interface, or are selected to be regenerated by the large model, and then the regenerated test cases are selected and confirmed; ​ The test case quality evaluation and reflection agent calculates a test case adoption rate, a requirement coverage rate and an executability, wherein the test case adoption rate is calculated as a proportion of a sum of a number of directly adopted test cases and a number of modified adopted test cases to a total number of generated test cases; the requirement coverage rate is calculated as a proportion of a number of testable requirement points covered by the generated test cases to a total number of testable requirement points; and the executability is calculated as a proportion of a number of test cases with directly executable conditions to a total number of test cases; and The test case quality is objectively evaluated by setting weights of the test case adoption rate, the requirement coverage rate and the executability, and fusing evaluation results of the test case adoption rate, the requirement coverage rate and the executability, to generate the complete test case set.

10. A test case generation system based on efficient collaboration of multiple agents, characterized in that, The method comprises the following steps: a multi-modal professional domain knowledge base construction module is configured to hierarchically classify, filter and normalize project development documents, software source code files, project test documents, software test document templates, expert review opinions, national / industry standards and typical defects of historical projects, to construct a multi-modal professional domain knowledge base; a task planning agent module is configured to generate test case generation tasks based on the project development documents and the software source code files of a to-be-tested project, and distribute the test case generation tasks to a test requirement analysis agent module; the test requirement analysis agent module is configured to extract test requirement points and descriptions of the test requirement points involved in a software configuration item based on the test case generation tasks, and label a corresponding tracking relationship between a test requirement point ID and a code snippet or a function in the software source code file, to construct a test requirement-code snippet / function set; a test case generation agent module is configured to generate test cases and test specification documents based on the test requirement-code snippet / function set and the multi-modal professional domain knowledge base.

Images