Automatic test case generation method and system based on multi-modal agent

Abstract

The invention provides a test case automatic generation method and system based on a multi-modal agent. The method comprises the following steps: receiving an original test case; analyzing the use case and extracting test operation elements through a pre-trained natural language processing model, wherein the model is obtained and configured with a hierarchical attention mechanism by adopting test field corpora through field adaptive training; based on the test operation elements, template matching is carried out in the business template knowledge base by adopting a hybrid matching algorithm combining semantic embedding vector similarity calculation and graph neural network topological relation analysis; performing data mapping in the test data knowledge base based on the test data entity; judging and generating preposed test data through a preposed scene recognition model constructed based on a conditional random field; and finally, assembling and outputting the structured test case for UI automatic test. According to the method, full-automatic, high-efficiency and high-quality conversion of the test case from the original description to the executable script can be realized, and the technical threshold and the maintenance cost for testers are reduced.

Description

Technical Field

[0001] This invention relates to the field of software test automation technology, and in particular to a method and system for automatically generating test cases based on multimodal intelligent agents. Background Technology

[0002] In software development, functional testing is a crucial step in ensuring product quality. While traditional manual testing is intuitive and flexible, it is inefficient, repetitive, and prone to errors, becoming a bottleneck restricting the improvement of development efficiency. To replace manual testing, automated user interface (UI) testing is gradually becoming an industry trend. However, building a sustainable and scalable UI automated testing system faces a core challenge: how to efficiently and accurately transform raw functional test cases into standardized, machine-executable test scripts.

[0003] Currently, the industry generally adopts two main implementation methods: one is for test developers to manually write automation scripts. Although this method can achieve precise control, it suffers from problems such as low efficiency, high skill threshold, and high maintenance costs, making it difficult to cope with the business needs of rapid iteration; the other is to generate test scripts based on "record-playback" tools. Although this method reduces coding requirements, the generated scripts are fragile, have poor stability, are not adaptable to UI changes, and have maintenance costs that are even higher than handwritten scripts, making it impossible to use at scale in continuous integration environments.

[0004] The aforementioned existing technologies have failed to effectively solve the problem of automating the conversion of test cases into scripts. The fundamental reason for this is the lack of a deep understanding of the semantics of test cases, as well as the lack of reusable and evolving business knowledge and test data support. Original test cases usually exist in the form of natural language or semi-structured text, and the process of converting them into executable scripts relies heavily on manual interpretation and experience-based judgment, resulting in low conversion efficiency, poor consistency, and maintenance difficulties. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and system for automatically generating test cases based on multimodal intelligent agents, in order to solve the problems of low efficiency in manual writing and conversion, high skill requirements for testers, frequent script failures due to frequent business changes, and high maintenance costs in the existing technology.

[0006] On one hand, the present invention provides a method for automatically generating test cases based on a multimodal agent, the method being executed by the multimodal agent, the method comprising: Receive the original test cases; The original test cases are parsed using a pre-trained natural language processing model to extract test operation elements. The natural language processing model is trained using domain-adaptive corpus and is configured with hierarchical attention mechanisms for recognizing operation verbs, associated interface objects, and extracting test data entities. The test operation elements include actions, interface objects, and test data entities. Based on the aforementioned test operation elements, a preset matching algorithm is used to perform business template matching in a pre-built business template knowledge base to obtain standardized business templates; wherein, the preset matching algorithm calculates the semantics of the test operation elements. The similarity between the embedding vector and the semantic embedding vector of each template in the business template knowledge base is determined, and the topological relationship between each business template node in the business template knowledge base is analyzed in combination with a pre-trained graph neural network to determine the matching result. Based on the test data entities, data mapping is performed in the pre-built test data knowledge base to obtain test data; The pre-trained pre-scenario recognition model determines whether the current test case needs to perform pre-data generation. If so, pre-test data is generated. The pre-scenario recognition model is built based on a conditional random field. The input features of the pre-scenario recognition model include dependent entities in the current test case, pre-dependency relationships between templates defined in the business template knowledge base, and scenario dependency patterns in historical execution logs. The standardized business template, the test data, and the pre-test data are assembled into structured test cases; Output the structured test cases for use in automated user interface testing.

[0007] In some embodiments of the present invention, the working process of the hierarchical attention mechanism includes: The action verbs in the original test cases are identified through the first attention layer; The identified action verbs are associated with at least one interface object through a second attention layer; The third attention layer extracts the corresponding test data entities from the text descriptions of the associated interface objects.

[0008] In some embodiments of the present invention, after determining whether the current test case needs to perform a pre-data generation operation through a pre-trained pre-scene recognition model, the method further includes: When it is determined that a preliminary data generation operation needs to be performed, the corresponding data generation rules are obtained from the test data knowledge base according to the type of the dependent entity. Based on the data generation rules, dynamic generation of prerequisite test data that satisfies the prerequisite dependencies is performed.

[0009] In some embodiments of the present invention, the method further includes a file processing procedure, which is executed in parallel with the determination of whether the current test case needs to perform a pre-data generation operation, including: Determine whether the current test case involves file operations; If file operations are involved, the corresponding test file will be matched or dynamically generated from the pre-built file library according to the file type and format requirements. Associate the matched or generated test files with the structured test cases.

[0010] In some embodiments of the present invention, the method further includes dynamically optimizing the business template knowledge base, wherein the optimization steps include: Obtain the execution result data of the user interface automated test on the structured test cases, the execution result data including the execution success rate and execution time; Based on the execution result data, the confidence score of the standardized business templates matched in the business template knowledge base is dynamically adjusted; When the confidence score is lower than a preset threshold, the corresponding business template is marked for optimization.

[0011] In some embodiments of the present invention, the optimization steps of the business template knowledge base further include: When the success rate of the structured test case is higher than a preset success threshold and the execution frequency is higher than a preset frequency threshold, the structured test case will be submitted for review. After approval, the structured test cases are converted into new business templates and added to the business template knowledge base.

[0012] In some embodiments of the present invention, the data mapping in the pre-built test data knowledge base includes: Bind the test data entity to at least one specific data instance pre-stored in the test data knowledge base; Alternatively, the test data entity can be associated with at least one predefined data generation rule in the test data knowledge base.

[0013] In some embodiments of the present invention, the assembly into structured test cases includes: The operation steps, test data, pre-test data, and related test file path information in the standardized business template are structurally assembled according to a preset format, and the integrity of the assembled structured data is verified; wherein, the preset format includes JSON format and XML format.

[0014] On the other hand, the present invention also provides an automated test case generation system based on multimodal intelligent agents, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of any of the methods mentioned above.

[0015] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method as described in any of the above submissions.

[0016] This invention provides a method and system for automated test case generation based on multimodal intelligent agents. By introducing a multimodal intelligent agent architecture, significant technical advancements and beneficial effects are achieved. By using domain-adaptive natural language processing parsing and hybrid matching algorithms, one-click generation of raw test cases into executable scripts is achieved, significantly improving processing efficiency and automation.

[0017] Meanwhile, the hybrid matching algorithm, which combines semantic understanding and graph structure analysis, can accurately understand the test intent and business logic context, handle complex and implicit test requirements, and generate test cases with high consistency and wide coverage.

[0018] Furthermore, by encapsulating the technical details required for UI automation within the intelligent agent, business testing experts can produce high-quality standardized test cases without having coding skills, significantly reducing the technical threshold and labor costs, and effectively solving the human resource bottleneck in the popularization of automation.

[0019] Furthermore, through a dynamic knowledge base optimization mechanism based on execution feedback (including confidence adjustment and new template accumulation), the system can continuously improve itself during use, fundamentally reducing script maintenance costs caused by business changes and achieving sustainable development of automated testing assets.

[0020] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0021] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of an automated test case generation method based on a multimodal intelligent agent in one embodiment of the present invention.

[0023] Figure 2 This is an overall architecture diagram of a test case automated generation method based on multimodal intelligent agents in one embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram illustrating the principle of a natural language processing model and a preset matching algorithm in one embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0026] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0027] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0028] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0029] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0030] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0031] To address the problems of low efficiency in manual script generation, high skill requirements for testers, frequent script failures due to frequent business changes, and high maintenance costs in existing technologies, this invention provides an automated test case generation method based on a multimodal intelligent agent. This method is executed by an intelligent agent pre-built and encapsulated with multiple artificial intelligence models and logic modules. Specifically, the intelligent agent includes a natural language processing model, a business template matching algorithm, a pre-defined scene recognition model, and interaction logic for various knowledge bases. Figure 1 As shown, the method includes the following steps S101~S107: Step S101: Receive the original test cases.

[0032] Step S102: Parse the original test cases using a pre-trained natural language processing model to extract test operation elements. The natural language processing model is trained using domain-adaptive corpus and is configured with hierarchical attention mechanisms for recognizing action verbs, associating interface objects, and extracting test data entities. Test operation elements include actions, interface objects, and test data entities.

[0033] Step S103: Based on the test operation elements, a preset matching algorithm is used to perform business template matching in a pre-built business template knowledge base to obtain standardized business templates. Specifically, the preset matching algorithm calculates the similarity between the semantic embedding vector of the test operation elements and the semantic embedding vectors of each template in the business template knowledge base, and combines this with a pre-trained graph neural network to analyze the topological relationships between business template nodes in the business template knowledge base to determine the matching result.

[0034] Step S104: Based on the test data entities, perform data mapping in the pre-built test data knowledge base to obtain test data.

[0035] Step S105: Determine whether the current test case needs to perform a pre-data generation operation using a pre-trained pre-scenario recognition model. If so, generate pre-test data. The pre-scenario recognition model is built based on a conditional random field. The input features of the pre-scenario recognition model include dependent entities in the current test case, pre-dependency relationships between templates defined in the business template knowledge base, and scenario dependency patterns in historical execution logs.

[0036] Step S106: Assemble standardized business templates, test data, and pre-test data into structured test cases.

[0037] Step S107: Output structured test cases for use in user interface automation testing.

[0038] like Figure 2 The diagram shown is the overall architecture of a method for automatically generating test cases based on multimodal intelligent agents.

[0039] In step S101, the system receives the original test cases. The original test cases refer to initial use case descriptions that have not yet been processed by the machine and are used to verify whether a specific function of the software is correct. These originate from test cases written by testers based on the product requirements document, or test cases automatically generated by a large language model based on the requirements description. Test cases are descriptions of the testing intent and expected behavior, not specific execution instructions.

[0040] In some embodiments, the original test cases are unstructured or semi-structured text.

[0041] In some embodiments, the system has a pre-configured receiving interface, which can be a file interface, an application programming interface (API), or a user interface (GUI). Specifically, the file interface can receive uploaded text files, such as .txt, .docx, and .xlsx format text files. The API interface receives test case data from other systems (such as a test management platform) via HTTP requests. GUI input allows users to directly paste or type text through a web interface or client interface.

[0042] In some embodiments, after receiving the original test cases, preprocessing operations are performed on them. These preprocessing operations typically include text cleaning, format recognition and normalization, etc. Specifically, text cleaning includes removing irrelevant characters, such as redundant spaces and garbled characters, and standardizing encoding formats. Format recognition and normalization includes recognizing the structure, extracting structured fields, and segmenting the text into sentences and paragraphs.

[0043] In step S102, the test cases processed by the above steps are input into the pre-trained natural language processing model for parsing, and the test operation elements of the test cases are extracted, specifically including actions, interface objects and test data entities.

[0044] In some embodiments, the construction and training of a natural language processing model specifically includes the following steps: The first stage is domain-adaptive training, aiming to enable the model to deeply understand the specialized terminology, expression habits, and business logic of the software testing domain, rather than remaining at the level of general language. Specifically: First, a massive amount of high-quality testing domain text data was collected to form a pre-training corpus. This testing domain text data includes at least historical test case libraries, software requirements specifications, user stories, test plans, defect reports, and other relevant documents.

[0045] Then, a general pre-trained language model is selected as the base model. The general pre-trained language model can be BERT, RoBERTa, or a similar model, which already has a powerful general language understanding capability.

[0046] Finally, the base model is trained using the constructed pre-trained corpus. In this process, the model learns to predict masked words in the test domain text, thereby transferring and adapting its original general knowledge to the test domain.

[0047] The second stage involves task-specific fine-tuning and integration of the hierarchical attention mechanism. In this embodiment, the hierarchical attention mechanism is integrated into the pre-trained natural language processing model (such as BERT) as an additional attention module. Specifically, three independent attention layers are stacked sequentially above the last hidden state output of the pre-trained model: the first attention layer, the second attention layer, and the third attention layer. Each attention layer employs a multi-head self-attention mechanism. The output of each layer is processed through LayerNorm and residual connections before serving as the input to the next layer. Finally, the output of the third attention layer passes through a fully connected layer and a Conditional Random Field (CRF) layer for sequence labeling tasks, such as labeling operation verbs, interface objects, and test data entities. During training, the entire model (including the pre-trained part and the newly added attention layers) undergoes end-to-end joint training, optimizing all parameters through backpropagation.

[0048] Specifically: First, labeled data is prepared for the "test operation element extraction" task. The training data consists of a large number of original test case sentences, and the words in each sentence are labeled with their identity: whether they are operation verbs, interface objects, or test data entities. For example, in the sentence "Enter username [test_user]", "enter" is labeled as an action, "username" is labeled as an interface object, and "test_user" is labeled as a test data entity.

[0049] Then, a designed hierarchical attention mechanism is superimposed on the pre-trained domain model as a task-specific output layer or internal enhancement structure.

[0050] Finally, labeled data is used to perform end-to-end supervised learning (i.e., fine-tuning) on ​​the entire model (including the domain-adaptive training part and the hierarchical attention layer). The model learns how to generate correct sequence labeling results based on the input sentence.

[0051] In some embodiments, the hierarchical attention mechanism is the core network structure of this natural language processing model, simulating the thought process of a test expert reading test cases: first find the action, then locate the object, and finally populate the data. The specific structure and workflow are as follows: The first attention layer is used to identify action verbs, focusing on words in the sentence that represent "actions". Specifically, it highlights key verbs by calculating the importance score (attention weight) of all words in the sentence to the concept of "action". This allows for the precise location of core action commands such as clicking, inputting, selecting, dragging, and verifying within the sentence.

[0052] The second attention layer is used to identify associated interface objects. Based on the identification of the action verb, the second attention layer is responsible for finding the interface elements directly related to this action. The attention weight calculation of this layer uses the action identified by the first layer as context, searching for the most relevant noun phrases to establish an "action-object" association. This allows for accurate identification of the object being operated on, such as login buttons, username input boxes, product checkboxes, and search results display areas, ensuring that automated operations can locate the correct user interface (UI) elements.

[0053] The third attention layer extracts the corresponding test data entities from the text descriptions of associated interface objects, i.e., identifies the specific data that needs to be input or verified. This layer combines the results of the first two layers (actions and objects) to focus on the words representing specific data values. Thus, it can accurately extract specific data such as test_user_01, ABC company, and 12345, or identify abstract data placeholders such as {username}, providing input for subsequent data mapping.

[0054] By using the three attention layers mentioned above, the triples (action, interface object, test data entity) can be accurately extracted to form structured test operation elements.

[0055] In step S103, based on the test operation elements, a preset matching algorithm is used to match business templates in a pre-built business template knowledge base to obtain standardized business templates. The preset matching algorithm employs a hybrid matching strategy, including two parts: similarity calculation based on semantic embedding and topological relationship analysis based on graph neural networks.

[0056] In some embodiments, similarity calculation based on semantic embedding includes the following steps: Using a semantic encoding model, such as Sentence-BERT, the textual descriptions of test operation elements and the descriptions of each template in the business template knowledge base are both converted into numerical vectors in a high-dimensional space, resulting in their respective semantic embedding vectors. In this vector space, texts with similar semantics have closer vector distances. Therefore, by calculating the cosine similarity between the vector of the test operation element (containing the test intent) and the vector of each business template, a preliminary matching score can be obtained. This score reflects the degree of similarity between the test operation element and each business template in terms of "literal meaning."

[0057] In some embodiments, topological relationship analysis based on graph neural networks includes the following steps: First, the business template knowledge base needs to be constructed as a graph structure, serving as the foundation for the graph neural network. Each business template represents a node, and business templates typically include user login templates, order creation templates, order payment templates, cancellation templates, and so on. Directed connections are established between nodes to represent the pre- and post-relationships or process sequences in the business logic. For example, the connection between the user login template and the order creation template indicates that logging in is required before an order can be created. The weight of each edge represents the strength or frequency of this dependency.

[0058] Based on the above settings, a graph structure is constructed, and the features of each node are initialized. The initial features are the semantic embedding vectors of the business template descriptions mentioned above.

[0059] Choose a graph neural network model as the base model, such as GCN or GraphSAGE.

[0060] The base model is trained to learn high-quality representations of nodes (business templates) in the graph. This representation includes not only the semantic information of the template itself but also its position and context within the overall business process graph. During training, a portion of the known edges in the graph are randomly masked (i.e., some business relationships are hidden), forcing the base model to predict the existence of these masked edges based on node features and the remaining graph structure. Through this process, the model is forced to learn how to infer potential relationships between nodes based on global topological information, thereby optimizing its internal parameters.

[0061] A well-trained graph neural network model can generate an enhanced representation vector for each business template node in the graph, which is the result of aggregating information from its neighboring nodes.

[0062] Finally, the semantic similarity score and the graph structure association score are weighted and summed, and the business template with the highest score is selected as the matching result. The semantic similarity score is obtained by calculating the cosine similarity between the semantic embedding vector of the test operation element and the semantic embedding vector of each business template in the business template knowledge base.

[0063] In some embodiments, the semantic similarity score and the graph structure association score are weighted and summed to obtain a comprehensive matching score, as shown in formula (1): ; (1) in, Indicates the overall matching score; This represents an adjustable weighting coefficient. Preferably, the initial value is set to 0.6, indicating a greater emphasis on semantic similarity. Represents semantic similarity score; The graph structure correlation score represents the graph structure correlation score.

[0064] The graph structure correlation score is obtained based on the following steps: The semantic embedding vector of the test operation element is regarded as a query node, which represents the test intent to be matched.

[0065] The feature vector (i.e., semantic embedding vector) of this query node is input together with the feature vectors of all business template nodes in the graph neural network into the pre-trained graph neural network (GNN). At this point, the graph structure is treated as an extended temporary graph during computation, and the query node is connected to all existing nodes in the graph through virtual edges to facilitate information propagation.

[0066] Through the message passing mechanism of graph neural networks, such as the attention-based graph attention network GAT layer, the attention weight between the semantic embedding vector of the test operation element and the feature vector of each business template node is calculated. This weight quantifies the association strength between the query node and each template node in the graph structure context.

[0067] All calculated attention weights are normalized (e.g., using the Softmax function), and the normalized weight values ​​are used as the graph structure correlation scores between query nodes and each business template node.

[0068] like Figure 3 The diagram shown illustrates the principle of the integrated natural language processing model and the preset matching algorithm.

[0069] In some embodiments, while matching standardized business templates, test case path processing is performed to determine the starting point of the test and prepare the test environment. The specific workflow includes: The specific access path of the function to be tested is parsed from the test operation elements, which is usually in the form of a URL.

[0070] The extracted access paths are then processed through path concatenation and parameterization, specifically: The relative path is concatenated with the base URL configured in the system to form a complete accessible address. If the access path contains dynamic parameters, the corresponding parameters are replaced with specific values ​​or variable names obtained from the test data knowledge base.

[0071] The processed complete path is transformed into explicit and executable pre-operation instructions and added to the beginning of the test case.

[0072] In step S104, while matching the standardized business template, data mapping is performed in the pre-built test data knowledge base based on the test data entity to obtain the test data.

[0073] Specifically, test cases are matched with various test data in a pre-built global test data knowledge base to obtain corresponding test data, providing a data foundation for subsequent testing operations. This includes: Based on the test data entities in the test operation elements, such as username, product ID, order amount, etc., determine which types of data are required for the current test case.

[0074] Based on the identified data type, a query is performed in a pre-built structured global test data knowledge base. The matching process employs a rule-based matching algorithm based on entity type and attribute constraints, including type matching, attribute constraint matching, priority selection, dynamically generated rules, and final binding return. Specifically: Type matching: Based on the type of the test data entity, such as username, order number, etc., search for data items of the same or compatible type in the knowledge base.

[0075] Attribute constraint matching: If the test data entity has attribute constraints, such as username length > 6 or order status = paid, then filter out data instances that meet all constraints.

[0076] Priority rules: If multiple data instances are matched, the data instance will be selected according to the preset priority, such as giving priority to the most recently generated data and the data marked as frequently used.

[0077] Dynamic rule generation: If no existing instance is found, data that meets the requirements will be dynamically generated based on the data generation rules associated with this type, such as regular expressions, function templates, etc.

[0078] Binding and Return: Bind the final determined data instance or generation rule to the test data entity and return it to the subsequent assembly module.

[0079] In step S105, the pre-trained pre-scenario recognition model determines whether the current test case requires pre-data generation. If so, pre-test data is generated. Here, a pre-scenario refers to the test data or system state that must be prepared in advance for the correct execution of the current test case. For example, assuming a "delete order" test is to be performed, a deleteable order must already exist in the system. This "existing order" is the pre-scenario.

[0080] Therefore, considering factors such as the inability of use cases to run independently and the existence of implicit dependencies between use cases, it is necessary to determine whether a pre-data generation operation is required.

[0081] The pre-context recognition model is a sequence labeling model built on Conditional Random Fields (CRFs). CRFs are discriminative probabilistic graphical models well-suited for label prediction of sequence data. Specifically, a test case can be viewed as a sequence of multiple test operation elements. The goal of this invention is to label each element in this sequence, determine whether it belongs to a "dependent entity," and thus infer whether the entire test case requires pre-context data generation. The advantage of using CRFs is that they consider the contextual information of the entire sequence, not just the current element, when making predictions.

[0082] The performance of the pre-scene recognition model depends on high-quality features. In this invention, the pre-scene recognition model receives the following three types of input features: dependent entities in the current test case, pre-dependencies between templates defined in the business template knowledge base, and scene dependency patterns in historical execution logs.

[0083] In this context, dependent entities in the current test case refer to test data entities that must exist in advance and are extracted by the natural language processing model. For example, in the "Delete order {order_id}" test case, "order" and "{order_id}" are labeled as "dependent entities".

[0084] The prerequisite dependencies between templates defined in the business template knowledge base refer to the logical dependencies between templates extracted from the graph structure of the business template knowledge base. For example, the "delete order template" depends on the "create order template" in business logic. This dependency relationship was defined when the knowledge base was built.

[0085] Scenario dependency patterns in historical execution logs refer to statistical patterns mined from the execution records of historical automated tests. For example, log analysis might reveal that if "test_case_A (create order)" is executed before "test_case_B (delete order)," then the success rate of B is 100%; otherwise, it is 0%. This forms a "scenario dependency pattern."

[0086] Using a large amount of labeled test case data (each test case is labeled with whether pre-data generation is required and which entities are dependent), combined with the three types of features mentioned above, the pre-scenario recognition model is trained through supervised learning. After training, the model can determine whether to perform pre-data generation based on the features of new test cases.

[0087] In some embodiments, while determining whether a preliminary data generation operation needs to be performed, file processing is carried out, including: Determine whether the current test case involves file operations. File operations refer to operations such as uploading and downloading.

[0088] If file operations are involved, the corresponding test files are matched or dynamically generated from the pre-built file library according to the file type and format requirements, and the matched or generated test files are associated with the structured test cases.

[0089] In some embodiments, when it is determined that a preliminary data generation operation needs to be performed, the corresponding data generation rules are retrieved from the test data knowledge base based on the type of the dependent entity. The type of the dependent entity can be an order, a user, or an uploaded file, etc. Then, based on the data generation rules, preliminary test data that satisfies the preliminary dependency relationships is dynamically generated.

[0090] In step S106, the standardized business template (including operation steps), test data (mapped, specific data values), and pre-test data (including dynamically generated data and environment that meet the pre-conditions) are assembled into structured test cases.

[0091] In some embodiments, structured test cases are in JSON or XML format.

[0092] In some embodiments, the assembled structured test cases are verified for completeness and correctness to improve data reliability.

[0093] In step S107, structured test cases are output for use in automated user interface testing.

[0094] In some embodiments, the method for automatically generating test cases based on multimodal intelligent agents further includes dynamically optimizing the business template knowledge base, including: After the downstream user interface automation test engine finishes executing the test cases generated by this invention, it collects detailed result data of the execution from the test execution platform or log system.

[0095] From massive amounts of data, execution success rate and execution time are extracted as evaluation metrics. Execution success rate is the percentage of test cases that successfully pass within a specific timeframe, representing all test cases generated based on a particular business template. Execution time is the average time or 95th percentile time taken to complete the execution of test cases generated based on a specific business template.

[0096] Based on the continuously collected execution success rate and execution time, the confidence score of each business template is dynamically calculated and updated using a preset algorithm formula. The new confidence score is then written into the business template knowledge base as an attribute of the corresponding template.

[0097] The system continuously monitors the confidence scores of all templates. Once a template's score falls below a preset threshold, its status is automatically marked as needing optimization, and the system is pushed to the maintenance personnel's to-do list or triggers an alarm notification.

[0098] In some embodiments, the system not only flags and repairs problematic old templates, but also automatically discovers and extracts new, valuable test patterns from successfully run test cases, transforming them into new standard templates, thereby expanding and enriching the knowledge base. Specifically: The system continuously monitors the execution data of all generated and executed structured test cases, selecting those with a success rate and execution frequency exceeding a preset threshold as excellent test cases. A high success rate indicates that the test case is stable and reliable, while a high execution frequency indicates that the business scenario covered by the test case is common and important.

[0099] The selected test cases are submitted to the review queue for review by test experts, or the system makes intelligent judgments based on more complex rules.

[0100] Replace the specific test data in the approved test cases with parameterized variables, and reorganize the operation steps, verification points, and other information according to the format required by the business template knowledge base.

[0101] Finally, the processed new template is added to the business template knowledge base and given an initial, high confidence score.

[0102] Corresponding to the above method, the present invention also provides an automated test case generation system based on multimodal intelligent agents, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method described above.

[0103] Corresponding to the above method, the present invention also provides an electronic device including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device performs the steps of the method as described above.

[0104] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned method. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0105] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0106] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0107] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically generating test cases based on multimodal intelligent agents, characterized in that, The method is executed by the multimodal agent, and the method includes: Receive the original test cases; The original test cases are parsed using a pre-trained natural language processing model to extract test operation elements. The natural language processing model is trained using domain-adaptive corpus and is configured with hierarchical attention mechanisms for recognizing operation verbs, associated interface objects, and extracting test data entities. The test operation elements include actions, interface objects, and test data entities. Based on the test operation elements, a preset matching algorithm is used to perform business template matching in a pre-constructed business template knowledge base to obtain standardized business templates. The preset matching algorithm calculates the similarity between the semantic embedding vector of the test operation elements and the semantic embedding vector of each template in the business template knowledge base, and combines a pre-trained graph neural network to analyze the topological relationship between each business template node in the business template knowledge base to determine the matching result. Based on the test data entities, data mapping is performed in the pre-built test data knowledge base to obtain test data; The pre-trained pre-scenario recognition model determines whether the current test case needs to perform pre-data generation. If so, pre-test data is generated. The pre-scenario recognition model is built based on a conditional random field. The input features of the pre-scenario recognition model include dependent entities in the current test case, pre-dependency relationships between templates defined in the business template knowledge base, and scenario dependency patterns in historical execution logs. The standardized business template, the test data, and the pre-test data are assembled into structured test cases; Output the structured test cases for use in automated user interface testing.

2. The method for automatically generating test cases based on multimodal intelligent agents according to claim 1, characterized in that, The working process of the hierarchical attention mechanism includes: The action verbs in the original test cases are identified through the first attention layer; The identified action verbs are associated with at least one interface object through a second attention layer; The third attention layer extracts the corresponding test data entities from the text descriptions of the associated interface objects.

3. The method for automatically generating test cases based on multimodal intelligent agents according to claim 1, characterized in that, After determining whether the current test case needs to perform pre-data generation operations through the pre-trained pre-scene recognition model, the process further includes: When it is determined that a preliminary data generation operation needs to be performed, the corresponding data generation rules are obtained from the test data knowledge base according to the type of the dependent entity. Based on the data generation rules, dynamic generation of prerequisite test data that satisfies the prerequisite dependencies is performed.

4. The method for automatically generating test cases based on multimodal intelligent agents according to claim 1, characterized in that, The method further includes a file processing procedure, which is executed in parallel with the determination of whether the current test case needs to perform a pre-data generation operation, including: Determine whether the current test case involves file operations; If file operations are involved, the corresponding test file will be matched or dynamically generated from the pre-built file library according to the file type and format requirements. Associate the matched or generated test files with the structured test cases.

5. The method for automatically generating test cases based on multimodal intelligent agents according to claim 1, characterized in that, The method also includes dynamically optimizing the business template knowledge base, the optimization steps of which include: Obtain the execution result data of the user interface automated test on the structured test cases, the execution result data including the execution success rate and execution time; Based on the execution result data, the confidence score of the standardized business templates matched in the business template knowledge base is dynamically adjusted; When the confidence score is lower than a preset threshold, the corresponding business template is marked for optimization.

6. The method for automatically generating test cases based on multimodal intelligent agents according to claim 5, characterized in that, The optimization steps for the business template knowledge base also include: When the success rate of the structured test case is higher than a preset success threshold and the execution frequency is higher than a preset frequency threshold, the structured test case will be submitted for review. After approval, the structured test cases are converted into new business templates and added to the business template knowledge base.

7. The method for automatically generating test cases based on multimodal intelligent agents according to claim 1, characterized in that, The data mapping in the pre-built test data knowledge base includes: Bind the test data entity to at least one specific data instance pre-stored in the test data knowledge base; Alternatively, the test data entity can be associated with at least one predefined data generation rule in the test data knowledge base.

8. The method for automatically generating test cases based on multimodal intelligent agents according to claim 1, characterized in that, The assembled structured test cases include: The operation steps, test data, pre-test data, and related test file path information in the standardized business template are structurally assembled according to a preset format, and the integrity of the assembled structured data is verified; wherein, the preset format includes JSON format and XML format.

9. A test case automated generation system based on multimodal intelligent agents, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images