Integrated test method and device and storage medium

Abstract

The embodiment of the invention provides an integrated testing method and device and a storage medium. In the embodiment of the invention, the AI large model, the MCP protocol and the multiple test frameworks of the execution layer are fused, so that an integrated test process with the cooperation of multiple test types is realized; the AI large model analyzes the test requirement document, recognizes multiple test requirements, is responsible for generating multiple test cases from the multiple test requirements, further converts the generated multiple test cases into multiple types of test scripts, calls a corresponding test framework to execute the test script corresponding to each test type through the MCP service of the execution layer, and performs the test on the test scripts. An integrated process from test case generation and script conversion to a test process is realized, and collaborative execution of multiple types of test scripts is realized. In addition, the integrated test framework further comprises a scheduling layer which is used for responding to the output of the AI large model, calling the MCP service of the execution layer, unloading the function of scheduling and managing the MCP service of the execution layer from the AI large model, and improving the reasoning efficiency of the AI large model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an integrated testing method, device and storage medium. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) in software development, automated testing tasks increasingly rely on the capabilities of LLMs, especially in UI (User Interface) testing. For example, users write test cases, LLMs perform content understanding on these test cases, and based on the understanding results, simulate user interactions with the application to complete automated testing.

[0003] However, the traditional testing process is fragmented and cannot support multiple test types, which limits the application of LLM in the testing field. Summary of the Invention

[0004] This application provides an integrated testing method, device, and storage medium to achieve an integrated testing process that integrates multiple testing types based on AI large models, MCP protocol, and multiple testing frameworks.

[0005] This application provides an integrated testing method applied to an integrated testing framework based on an AI large model and the MCP protocol, including a model layer and an execution layer. The method includes: obtaining a test requirement document for a target application; inputting the test requirement document into an AI large model located on the model layer, so that the AI ​​large model performs the following operations under the guidance of a first model prompt: performing multi-intent recognition on the test requirement document to obtain multiple test requirements; different test requirements include different test types and test objectives, and different test types are adapted to different test frameworks; generating multiple test cases based on the test types and test objectives included in the multiple test requirements, combined with the target information of the target application, with different target information corresponding to different test types; converting the multiple test cases into multiple test scripts that can be executed by the multiple test frameworks according to the multiple test types; and executing the multiple test scripts by calling the multiple test frameworks through multiple first MCP services located on the execution layer based on the MCP protocol; wherein, different first MCP services integrate calling interfaces for different test frameworks for calling different test frameworks.

[0006] This application also provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor is coupled to the memory to execute the computer program to implement the steps in the various methods provided in this application.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above methods.

[0008] This application also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, enable the processor to implement the steps in the methods described above.

[0009] In this embodiment, an integrated testing process for multiple test types is achieved by integrating an AI big data model with multiple test frameworks in the execution layer and the MCP protocol. The AI ​​big data model parses the test requirement document, identifies multiple test requirements, and is responsible for generating multiple test cases from these requirements. Then, it converts the generated test cases into test scripts of various types. The corresponding test scripts for each test type are executed by calling the corresponding test framework through the MCP service in the execution layer. This achieves an integrated process from test case generation and script conversion to the testing process, and enables the collaborative execution of multiple test scripts.

[0010] Furthermore, in an optional embodiment of this application, the integrated testing framework also includes a scheduling layer located between the model layer and the execution layer. The scheduling layer includes a target MCP service, which is used to respond to the output of the AI ​​large model and call the MCP service of the execution layer, so as to offload the function of scheduling and managing the MCP service of the execution layer from the AI ​​large model to the scheduling layer, thereby improving the inference efficiency of the AI ​​large model. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0012] Figure 1 A flowchart illustrating an integrated testing method provided for an exemplary embodiment of this application;

[0013] Figure 2 This is a flowchart illustrating an integrated testing method according to another exemplary embodiment of this application;

[0014] Figure 3 A flowchart illustrating an integrated testing method provided as another exemplary embodiment of this application;

[0015] Figure 4 A schematic diagram of an integrated testing framework provided for another exemplary embodiment of this application;

[0016] Figure 5 A flowchart illustrating an integrated testing method provided as another exemplary embodiment of this application;

[0017] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0020] Additionally, it should be noted that when user interaction operations or triggering operations are involved in the embodiments of this application, these operations include, but are not limited to, various interaction methods such as touch operations, gesture operations, voice operations, head movement operations, and eye movement operations. Touch operations include, but are not limited to, click operations, double-click operations, long-press operations, swipe operations, pinch operations, or mouse hover operations. Swipe operations include, but are not limited to, straight-line swipes and curved-line swipes.

[0021] Furthermore, it should be noted that, in the cases where the embodiments of this application involve jumping between the first interface and the second interface, the jumping methods involved in the embodiments of this application include, but are not limited to: jumping directly from the first interface to the second interface, or jumping from the first interface to the task interface and completing the corresponding task operation on the task interface before jumping to the second interface; completing the corresponding task operation on the task interface includes, but is not limited to: completing the game operation on the game interface when the task interface is implemented as a game interface; completing identity authentication on the identity authentication interface when the task interface is implemented as an identity authentication interface; completing the recharge operation on the recharge interface when the task interface is implemented as a recharge interface; and so on.

[0022] To address the technical issues of fragmented traditional testing processes and their inability to support multiple test types, this application embodiment integrates multiple test frameworks at the execution layer with an AI big data model and the MCP protocol to achieve a unified testing process that enables collaboration across multiple test types. The AI ​​big data model parses test requirement documents, identifies various test requirements, and is responsible for generating multiple test cases from these requirements. It then converts these generated test cases into test scripts of various types. The execution layer's MCP service calls the corresponding test framework to execute the test scripts for each test type, achieving a unified process from test case generation and script conversion to the testing process, and enabling the collaborative execution of multiple test script types.

[0023] Furthermore, in an optional embodiment of this application, the integrated testing framework also includes a scheduling layer located between the model layer and the execution layer. The scheduling layer includes a target MCP service, which is used to respond to the output of the AI ​​large model and call the MCP service of the execution layer, so as to offload the function of scheduling and managing the MCP service of the execution layer from the AI ​​large model to the scheduling layer, thereby improving the inference efficiency of the AI ​​large model.

[0024] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0025] like Figure 1 As shown in this embodiment, an integrated testing framework based on a large AI model and the MCP (Model Control Protocol) is provided. The integrated testing framework includes a model layer and an execution layer. The AI ​​model in the model layer is interconnected with the MCP service in the execution layer. Specifically, the large AI model is interconnected with the MCP service (Server) in the execution layer based on the MCP protocol, enabling the large AI model to schedule the MCP service located in the execution layer based on the MCP protocol.

[0026] In this embodiment, a large AI model refers to a model whose parameters meet the set parameter quantity requirements, without any limitation on the parameter quantity requirements. Parameter quantity requirements can vary in different scenarios and domains. For example, in some scenarios or domains, a large AI model refers to a model with a parameter scale reaching tens of billions, hundreds of billions, or even trillions; this definition of the model's parameter quantity is merely an example.

[0027] In this embodiment, the MCP service can also be referred to as a test plugin. The MCP service is an extensible plugin developed in accordance with the MCP protocol. The MCP protocol provides a standardized interface for connecting large AI models to various tools. Based on the standardized interface of the MCP protocol, plug-and-play functionality between AI models and various tools is achieved without the need to develop a separate adapter for each tool.

[0028] In this embodiment, the execution layer includes multiple MCP services, which are built based on the MCP protocol. Each MCP service in the execution layer encapsulates the calling interface of a test framework. Each MCP service integrates the calling interface of a corresponding test framework, and different MCP services integrate calling interfaces of different test frameworks for invoking different test frameworks. The number of MCP services in the execution layer can be dynamically changed. When a new test framework appears or needs to be added, an MCP service adapted to the new test framework can be generated and deployed to the node devices in the execution layer. When a test framework is no longer needed or is phased out due to version aging, the corresponding MCP service can be uninstalled, enabling dynamic deployment and updating of MCP services.

[0029] The following describes the integrated testing method provided in the embodiments of this application, which is applied to the aforementioned integrated testing framework.

[0030] like Figure 1 As shown, in this embodiment, a test requirement document for the target application is obtained, and the test requirement document is input into the large AI model located on the model layer. The large AI model executes the test steps under the guidance of the first model prompt.

[0031] In this embodiment, the method of obtaining the test requirement document for the target application is not limited. For example, it could be receiving a test requirement document uploaded by a user; or receiving an address link corresponding to the test requirement document uploaded by a user, and then obtaining the test requirement document through that address link.

[0032] The target application refers to the application that needs to be tested, such as shopping applications, video applications, live streaming applications, game applications, language processing applications, etc. Optionally, the target application can be an application with a graphical user interface.

[0033] In this embodiment, the product form of the target application is not limited. For example, the product form of the target application can be a web page; or it can be a standalone APP (Application); or it can be a mini-program that depends on the APP to run; or it can be an interface; or it can be a microservice.

[0034] In this embodiment, the test requirements document is a document used to describe the test requirements of the target application in terms of UI, interface, or performance. The test requirements document can be written in natural language. For example, it may include: test background, test objectives, the scope of test types, and test environment, etc., without limitation. Optionally, the test requirements document may also describe relevant information about the target application, such as its type, functions, and description, to provide richer information for generating test cases.

[0035] In this embodiment, if the test requirements document is obtained, such as Figure 1 As shown, the test requirements document is input into the large AI model located at the model layer. Upon receiving the test requirements document, the AI ​​model executes an integrated testing process for the target application, guided by the first model prompt.

[0036] In this embodiment, the method of providing the first model prompt is not limited. For example, the first model prompt can be constructed outside the large AI model using a prompt template. In this case, the first model prompt and the test requirements document are input into the large AI model together. Alternatively, the first model prompt can be learned and integrated into the large AI model during training, becoming part of the large AI model. This eliminates the need to construct the model prompt from an external source using a prompt template each time the model is called, improving the convenience and efficiency of model invocation.

[0037] In this embodiment, as Figure 1 As shown, the AI ​​model performs multi-intent recognition on the test requirements document, identifying various test requirements. Multi-intent recognition refers to identifying the corresponding test type and test objective for different semantic fragments in the test requirements document, thereby obtaining multiple test requirements. Each test requirement includes a test type and a test objective. The test objective is used at least during test execution to compare with the execution result to determine whether the execution result meets the test objective. Different test requirements include different test types and test objectives, and different test types are adapted to different test frameworks.

[0038] In this embodiment, the specific implementation of the test type is not limited. Test types include, but are not limited to, unit tests, integration tests, UI tests, API (application program interface) tests, and performance tests. Multiple first MCP services can expose call interfaces for test frameworks adapted to different test types to the large AI model.

[0039] In this embodiment, a testing framework refers to a collection of software tools that support automated testing for specific test types. The main differences between different testing frameworks lie in the types of tests they support, the programming languages ​​they use, and the test script templates they employ.

[0040] The multiple first MCP services include, but are not limited to: a first MCP service integrating a test framework for UI testing, a first MCP service integrating a test framework for API testing, and a first MCP service integrating a test framework for performance testing. In one example, UI testing uses the "Selenium" test framework, API testing uses the "Pytest" test framework, and performance testing uses the "JMeter" test framework, but this does not constitute a limitation on this embodiment.

[0041] Furthermore, such as Figure 1 As shown, the AI ​​model generates multiple test cases based on various testing requirements, including test types and objectives, and in conjunction with the target application's target information. Target information refers to relevant information about the target application used to assist in test case generation, such as, but not limited to, UI information, interface information, functional information, and runtime status information. Different test types require different target information. For example, UI testing requires target information including, but not limited to, the target application's UI elements, interface layout, and other UI information, as well as the target application's functional information; API testing requires target information including, but not limited to, the interface information called by the target application and the target application's functional information; performance testing requires target information including, the target application's runtime status information and the interface information called.

[0042] In this embodiment, a test case refers to a structured description generated based on test requirements, including but not limited to: input parameters, test objectives, and execution paths. An execution path refers to the execution path corresponding to each test type in the integrated testing process; different execution paths map to different test types and their corresponding test frameworks. For example, the execution path for UI testing maps to UI-related test frameworks, and the execution path for API testing maps to API-related test frameworks.

[0043] Furthermore, such as Figure 1As shown, based on multiple test frameworks corresponding to various test types, multiple test cases are converted into multiple test scripts that can be executed by the various test frameworks. In this embodiment, for multiple test cases, the corresponding multiple test types are identified, and then for each test type, the test cases are converted into test scripts by combining the input parameters, test objectives, assertion conditions, etc.

[0044] In this embodiment, as Figure 1 As shown, based on the MCP protocol, multiple first MCP services located in the execution layer call multiple test frameworks to execute multiple test scripts. Different first MCP services integrate calling interfaces for different test frameworks. For any given test script, a first MCP service is scheduled, and the first MCP service calls the corresponding test framework through its integrated test framework calling interface to run the test script, thus testing the target application. The multiple first MCP services are at least a portion of the MCP services deployed in the execution layer. The number of first MCP services is the same as the number of test types identified from the test requirements document. Upon receiving a test script, the first MCP service starts the runtime environment of the corresponding test framework, provides the test script to the test framework, and the test framework loads and runs the test script in the provided runtime environment. Furthermore, the first MCP service can also monitor the runtime status of the test framework and collect execution logs during the execution of the test script. This information can be fed back to the AI ​​big data model to help the AI ​​big data model locate the root cause of test failures.

[0045] Optionally, the first MCP service and the testing framework can be deployed on the same server or in a distributed manner on different servers; there is no limitation on this.

[0046] In this embodiment, an integrated testing process for multiple test types is achieved by integrating an AI big data model with multiple test frameworks in the execution layer and the MCP protocol. The AI ​​big data model parses the test requirement document, identifies multiple test requirements, and is responsible for generating multiple test cases from these requirements. Then, it converts the generated test cases into test scripts of various types. The corresponding test scripts for each test type are executed by calling the corresponding test framework through the MCP service in the execution layer. This achieves an integrated process from test case generation and script conversion to the testing process, and enables the collaborative execution of multiple test scripts.

[0047] In an alternative embodiment, such as Figure 2As shown, the aforementioned integrated testing framework also includes a scheduling layer located between the model layer and the execution layer. The scheduling layer is responsible for scheduling and managing the MCP services of the execution layer. On the one hand, it offloads the functions of scheduling and managing the MCP services of the execution layer from the AI ​​large model to the scheduling layer, allowing the AI ​​large model to focus more on its inference capabilities and improve its inference efficiency. On the other hand, when expanding the testing framework, only the target MCP service on the scheduling layer needs to be extended, without modifying the AI ​​large model, which has high flexibility and provides conditions for the flexible expansion of the testing framework.

[0048] In this embodiment, as Figure 2 As shown, the scheduling layer includes a target MCP service. The target MCP service is used to respond to the scheduling task instructions of the AI ​​large model output, and to call the MCP service of the execution layer according to the scheduling task instructions.

[0049] Based on this, in an optional embodiment, when multiple test frameworks are scheduled to execute multiple test scripts through multiple first MCP services located in the execution layer, such as Figure 2 As shown, the AI ​​big model can generate a first scheduling task instruction based on multiple test types and input the first scheduling task instruction into the target MCP service. The target MCP service receives the first scheduling instruction and, guided by the second model prompt, selects multiple first MCP services that are compatible with the multiple test types from multiple MCP services located in the execution layer according to the first scheduling task instruction, and returns the access links of the selected multiple first MCP services to the AI ​​big model.

[0050] In this embodiment, the target MCP service is primarily responsible for: matching and returning multiple suitable first MCP services from the execution layer MCP services according to the test type, and returning the access link to the AI ​​large model. Optionally, the target MCP service matches the execution layer MCP services with the test type carried in the first scheduling task instruction, selects one or more first MCP services, and returns the access link. If no match is found, i.e., the execution layer MCP service cannot provide the corresponding service, a lightweight execution environment is started in the target MCP service. This lightweight execution environment can temporarily host and run the missing MCP service, i.e., starting the "last fallback" mode for the target MCP service, thereby ensuring that the current test process is not interrupted. For details on the "last fallback" mode, please refer to the subsequent embodiments.

[0051] Furthermore, such as Figure 2As shown, after receiving access links to multiple first-level MCP services returned by the scheduling layer, the AI ​​large model adds these access links to its MCP tool list. In this way, the AI ​​large model can treat these multiple first-level MCP services as directly invoked external tools. Furthermore, the AI ​​large model can access these multiple first-level MCP services based on their access links and multiple test scripts, thereby invoking multiple test frameworks to execute multiple test scripts through these services.

[0052] Optionally, when the AI ​​large model accesses the first MCP service via an access link, it retrieves the corresponding access link from the MCP tool list, accesses the corresponding first MCP service according to the access link, and passes the relevant test scripts to the accessed first MCP service. The first MCP service then calls its integrated test framework, executes the received test scripts, and returns the execution results to the AI ​​large model. This enables the AI ​​large model to uniformly call multiple test frameworks through the MCP protocol.

[0053] Further optional, such as Figure 3 As shown, execution paths for different test types can be executed concurrently. For example, execution paths include, but are not limited to: UI test-related execution paths, API test execution paths, and performance test execution paths. Refer to the foregoing examples for the definition of execution paths.

[0054] In one optional embodiment, the target MCP service of the scheduling layer and the MCP service of the execution layer can be deployed in multiple locations to achieve disaster recovery. Further optionally, in order to ensure the normal execution of the test, the target MCP service provides mechanisms such as fault detection, dynamic deployment in case of failure, migration and hot upgrade, to ensure that the required execution layer MCP service can be repaired, expanded, migrated or upgraded in a timely manner.

[0055] Optionally, the target MCP service integrates a Test Task Scheduler. The Test Task Scheduler is responsible for rationally arranging the execution order of multiple test scripts according to test requirements and the resource status of the execution layer MCP service. It supports functions such as concurrency, priority control, and load balancing, thereby improving the execution efficiency of test scripts and the resource utilization of the execution layer MCP service.

[0056] Optionally, the target MCP service integrates a test execution engine. The test execution engine calls the first MCP service in the execution layer to execute multiple test scripts through multiple first MCP services calling multiple test frameworks, forming a unified execution control entry point and facilitating the collection and management of test results.

[0057] In one optional embodiment, the MCP service located in the execution layer is divided into multiple service domains, each containing MCP services that provide different services. By dividing into multiple service domains, different types of MCP services form clear boundaries, avoiding coupling between different functions. Furthermore, the AI ​​large model can more efficiently match MCP services from the corresponding service domains during scheduling, reducing scheduling complexity. In addition, different service domains can be independently extended and maintained, achieving modular management.

[0058] Multiple service domains include a first service domain, which comprises multiple MCP services providing testing services by running different testing frameworks. Guided by a second model prompt, the target MCP service selects the first service domain from the multiple service domains according to a first scheduling task instruction, and selects multiple first MCP services from the multiple MCP services included in the first service domain. The identifier of the first service domain and the access links of the multiple first MCP services are returned to the AI ​​big model. Optionally, when selecting the first MCP service from multiple MCP services, for any given test script, if multiple MCP services can run the test script, the selection can be based on the access popularity, service quality, and / or load of each MCP service, prioritizing the MCP service with higher access popularity, better service quality, or lighter load as the first MCP service. For example, the service quality of an MCP service can be statistically obtained from the results of executing each test script. For instance, the service quality of an MCP service can be calculated by statistically analyzing the execution success rate of the executed test scripts over a recent period; a higher execution success rate indicates higher service quality. Alternatively, the service quality of an MCP service can be determined by statistically analyzing the execution time or efficiency of the test scripts executed by the MCP service. Higher execution efficiency indicates higher service quality. Alternatively, the service quality of an MCP service can be determined by combining both the success rate and efficiency of its test script execution. Statistical analysis of the service quality of an MCP service can be performed by the target MCP service itself.

[0059] Furthermore, in this optional embodiment, when adding access links for multiple first MCP services to the MCP tool list of the AI ​​large model, and accessing multiple first MCP services according to the access links for multiple first MCP services and multiple test scripts to call multiple test frameworks to execute multiple test scripts through multiple first MCP services, the method includes: adding the identifier of the first service domain and the access links for multiple first MCP services to the corresponding MCP tool list of the AI ​​large model, and accessing multiple first MCP services according to the identifier of the first service domain, the access links for multiple first MCP services and multiple test scripts to run multiple test frameworks to execute multiple test scripts through multiple first MCP services.

[0060] Further optionally, the multiple service domains also include: a second service domain and / or a third service domain, wherein the second service domain includes a second MCP service responsible for generating test cases, and the third service domain includes a third MCP service for converting test cases into test scripts.

[0061] In this embodiment, when generating multiple test cases based on various test requirements, including test types and test objectives, and combined with the target information of the target application, the process includes: generating a second scheduling task instruction based on various test types and multiple test objectives; inputting the second scheduling task instruction into the target MCP service, so that the target MCP service, guided by the second model prompt, selects a second service domain from multiple service domains according to the second scheduling task instruction, and returns the access link of the second MCP service in the second service domain to the AI ​​big model; the AI ​​big model adds the access link of the second MCP service to the MCP tool list of the AI ​​big model, and in the process of generating multiple test cases, calls the second MCP service according to the access link of the second MCP service to assist in generating multiple test cases.

[0062] In this embodiment, during the generation of multiple test cases, the second MCP service can be invoked in stages such as intent recognition and test case generation.

[0063] In this embodiment, when the AI ​​large model performs multi-intent recognition on the test requirement document, in order to more accurately identify the test requirements in the document, the auxiliary parsing interface provided by the second MCP service can be called to improve the accuracy of understanding specific scenarios and domain contexts.

[0064] In this embodiment, the second MCP service can also be used to assist in the generation of test cases. The main purpose of the assisted generation is to offload some of the rule-based and structured test case generation steps from the AI ​​big model to the second MCP service for execution, rather than relying entirely on the AI ​​big model's inference generation. This reduces the burden on the AI ​​big model in some test case generation steps and improves test case generation efficiency. For example, if the test type is API testing, the AI ​​big model obtains the API interface definition and parameter rules provided by the second MCP service by calling the second service domain. Based on the API interface definition and parameter rules, the AI ​​big model transforms the natural language description into structured API test cases.

[0065] In this embodiment, when converting multiple test cases into multiple test scripts that can be executed by multiple test frameworks corresponding to multiple test types, the process includes: generating a third scheduling task instruction based on multiple test types; inputting the third scheduling task instruction into the target MCP service, so that the target MCP service, guided by the second model prompt, selects a third service domain from multiple service domains according to the third scheduling task instruction, and returns the access link of the third MCP service in the third service domain to the AI ​​big model; the AI ​​big model adds the access link of the third MCP service to the MCP tool list of the AI ​​big model, and in the process of converting multiple test cases into multiple test scripts, calls the third MCP service according to the access link of the third MCP service to perform auxiliary conversion of multiple test scripts.

[0066] In this embodiment, when the AI ​​large model needs to convert API test cases into test scripts that can be executed on a specific testing framework, it can call the third MCP service of the third service domain. The third MCP service utilizes the script templates of the specific testing framework integrated within it to achieve the conversion from test cases to runnable test scripts. In an optional embodiment, the AI ​​large model generates an initial test script, and then calls the script optimization service provided by the third MCP service to perform structural optimization, robustness enhancement, and other processing on the initial test script to generate the final test script.

[0067] In this embodiment, the main purpose of calling the third MCP service to assist in the conversion is to perform rule-based conversion operations related to the test framework adaptation, thereby assisting in the generation of multiple test cases into multiple test scripts. This allows some rule-based conversion operations to be offloaded from the large AI model to the third MCP service for execution, thus improving conversion efficiency.

[0068] In an optional embodiment, the purpose of the target MCP service involves two aspects. The first aspect refers to the scheduling and management of the execution layer MCP service described in the above embodiments. The second aspect refers to starting a lightweight execution environment, such as a sandbox or container, when the execution layer MCP service cannot provide the corresponding service. The missing MCP service can be temporarily hosted and run in this lightweight execution environment, that is, starting the "last resort" mode for the target MCP service, thereby ensuring that the current test process is not interrupted.

[0069] Based on this, in an optional embodiment, if it is identified that the target MCP does not return the access link of the first MCP service corresponding to the first test type, and the first test type is located within the set of test types, the access link of the target MCP service is added to the MCP tool list of the AI ​​large model; for the first test type, according to the access link of the target MCP service, the target MCP service is called so that the target MCP service can call the test framework adapted to the first test type to execute the test script corresponding to the first test type; the target MCP service integrates the calling interface of the test framework corresponding to each test type in the set of test types, and the first test type is any test type among multiple test types.

[0070] The process of the target MCP service calling a test framework adapted to the first test type to execute the test script corresponding to the first test type includes: starting a lightweight execution environment; activating the call interface of the test framework corresponding to the first test type in the lightweight execution environment; calling the test framework adapted to the first test type through the call interface; writing the test script into the memory space corresponding to the test framework; and loading and running the test script from the memory space after the test framework starts.

[0071] In this embodiment, the preset test type set is a pre-defined set of key test types (such as UI testing, API testing, and performance testing). By pre-setting the key test type set, the test process of core test types can be ensured to run uninterrupted in a lightweight execution environment. Furthermore, the preset key test type set supports the flexible setting of test types used in the "last resort mode," rather than using all test types as a fallback, thereby reducing the burden on the target MCP service and achieving a balance between service reliability and resource consumption.

[0072] In one optional embodiment, the large AI model is associated with a pre-built knowledge base; the knowledge base includes test annotation fragments corresponding to various test types, and each test type's corresponding test annotation fragment corresponds to a different test requirement type tag. The process of performing multi-intent recognition on the test requirement document to obtain various test requirements includes: performing semantic structure parsing on the test requirement document to obtain the hierarchical relationship between titles and paragraphs in the test requirement document; constructing a semantic structure tree based on this hierarchical relationship; determining candidate test requirement fragments for multi-intent recognition from the test requirement document based on the semantic structure tree, and expanding the context window of the candidate test requirement fragments; semantically matching the candidate test requirement fragments after expanding the context window with the test annotation fragments in the knowledge base to obtain a set of matched test requirement type tags; and obtaining various test requirements based on the set of test type tags.

[0073] In this embodiment, the hierarchical relationship between titles and paragraphs is identified through semantic analysis and organized into a tree structure, namely a semantic structure tree.

[0074] In this embodiment, the AI ​​big model is associated with a pre-built knowledge base, which is a pre-accumulated experience knowledge base in the testing domain. Based on the knowledge base, multiple types of testing requirements are identified. During the identification process, a semantic structure tree is constructed by combining the hierarchical structure of titles and paragraphs, context window expansion, and multi-label classification to improve the accuracy of identifying multiple types of testing requirements.

[0075] Optionally, when the test type is UI testing, before converting the test cases into test scripts, obtain the application interface screenshots corresponding to the test requirements document; perform interface element recognition on the application interface screenshots to identify the text information and location information of the interactive interface elements; based on the text information and location information of the interactive interface elements, construct structured interface element knowledge entries and store them in the knowledge base.

[0076] Accordingly, when converting multiple test cases into multiple test scripts that can be executed by multiple test frameworks corresponding to multiple test types, the process includes: matching the corresponding interface element knowledge entries from the knowledge base when the operation description of any test case involves the operation description of the application interface; and converting the multiple test cases into multiple test scripts that can be executed by multiple test frameworks based on the text information and position information of interactive interface elements in the matched interface element knowledge entries.

[0077] In one example, when the test requirements document includes the description "perform UI testing of the login function," the AI ​​model can obtain a screenshot of the login interface of the target application and perform interface element recognition on the screenshot to obtain the text and location information of interactive interface elements such as the "login button" and "username input box." Furthermore, based on the text and location information of these interactive interface elements, interface element knowledge entries are constructed and stored in a knowledge base. During the process of converting test cases into test scripts, when the operation description in the test case involves "clicking the login button and entering the username," the AI ​​model will match the corresponding interface element knowledge entry from the knowledge base and convert it into part of the test script executable by the test framework. By leveraging the pre-stored text and location information of interactive interface elements in the interface element knowledge entries, ambiguity that might arise from using natural language descriptions in the test requirements document is avoided, resulting in more accurate generation of test scripts adapted to the test framework.

[0078] Further optional, such as Figure 4As shown, the large AI model includes multiple parallel channels, such as channels b1 to bN, where N is greater than or equal to 2. Each channel has a corresponding test type label and is configured with a test case template for that test type. Specifically, the test type label for channel b1 is a1, for channel b2 it is a2, and for channel bN it is aN. Furthermore, Figure 4 The execution layer is divided into the first service domain, the second service domain, and the third service domain.

[0079] In this embodiment, when generating multiple test cases based on various test requirements, including test types and test objectives, and combined with the target information of the target application, the process includes: matching multiple test types with the test type tags corresponding to each channel to determine the channel suitable for each test requirement; and generating multiple test cases using the test case templates on the channels suitable for each test requirement and historical test cases. Each channel has a test case generation module, which can call a second MCP service to assist in test case generation. Multiple parallel channels are obtained during the training of the large AI model based on the results of classifying training samples, and during the iteration of the large AI model, new channels and their corresponding test type tags are generated based on new samples, thereby achieving dynamic expansion of channels and improving test type coverage.

[0080] Optionally, an isolated architecture can be adopted in the channel design, with each channel having exclusive GPU (Graphics Processing Unit) computing slices or thread pools, and runtime isolation can be achieved through containerization (Docker) or virtualization. For example, each channel is deployed in an independent container, and each channel runs in its own independent container environment, with containers isolated from each other. Since each channel has exclusive GPU computing slices or thread pools, multiple channels can run in parallel without interference between different channels. Multiple test types can be quickly mapped to the corresponding channels for test case generation, avoiding the performance bottleneck caused by serial processing in a single channel and improving test case generation efficiency.

[0081] Optionally, the AI ​​big model supports dynamically loading new channels via a hot-swapping mechanism. When a new test type is identified in the test requirements document, the AI ​​big model generates new test requirement tags and their corresponding channel parameters for the new test type, and constructs a new channel based on these parameters. The new channel is then attached to the parallel channels to smoothly expand the processing capacity for the new test type without interrupting the execution of existing channels. Correspondingly, channels can also be canceled when a test type is canceled.

[0082] Further optional, such as Figure 3As shown, the above method also includes: generating execution logs during the execution of the test script; in the event of execution failure, returning the execution logs to the AI ​​big model through the first MCP service, so that the AI ​​big model can analyze the cause of the execution failure based on multiple dimensions of information, obtain the analysis results, generate repair suggestions based on the analysis results, and repair the information in the corresponding dimensions based on the repair suggestions.

[0083] In this embodiment, the target MCP service receives execution logs from different first MCP services and, guided by the second model prompt, returns these execution logs to the AI ​​big model. The AI ​​big model performs semantic parsing on the execution logs and, combined with the target application's target information (such as UI information, interface information, and runtime status information), analyzes the reasons for failure from multiple dimensions. Based on the analysis results, the AI ​​big model generates repair suggestions. Optionally, the multiple dimensions of information include, but are not limited to, test script dimensions, test case dimensions, and test environment dimensions. In an optional embodiment, when repairing information in the corresponding dimensions based on the repair suggestions, the test script can be repaired based on the repair suggestions. Subsequently, the AI ​​big model distributes the repair suggestions to the corresponding first MCP service through the target MCP service. The first MCP service runs the repaired test script to complete the closed-loop repair. This repair process can be repeated multiple times until a preset number of attempts is reached, at which point a manual interface is invoked for manual processing.

[0084] In another optional embodiment, when repairing information in the corresponding dimension based on the repair suggestions, it is also possible to check whether the test environment matches the test script based on the repair suggestions. If they do not match, a new first MCP service adapted to the test script is selected from multiple first MCP services in the execution layer, and the adapted first MCP service is used as the new test environment for executing the test script. In this embodiment, an AI large model is introduced to perform semantic understanding of the execution logs of execution failures, quickly locating the root cause of the problem. Furthermore, repair suggestions are automatically generated and repairs are performed in the corresponding dimension based on the repair suggestions, improving debugging efficiency.

[0085] In one optional embodiment, determining the method of test script execution failure includes: comparing the execution result of the test script with the corresponding test target; if the execution result does not meet the test target, then determining that the test script execution has failed.

[0086] In one optional embodiment, execution logs are collected in real time during the testing process, and a large AI model is used for semantic-level parsing to extract information such as key events, anomaly patterns, and performance bottlenecks. Furthermore, based on the semantic parsing results of the execution logs, the large AI model automatically generates a test report combining text and graphics. The test report includes a test summary, failure details, and remediation suggestions for easy review and decision-making. Optionally, the test report is output in natural language and supports clear explanations of the reasons for test failures to facilitate understanding by non-technical personnel.

[0087] In an optional embodiment, an AI testing assistant for triggering the aforementioned integrated testing process can be embedded in the sidebar or floating window of the test requirements document. This AI testing assistant can respond to the user's triggering operation and realize "one-click testing within the document".

[0088] In one alternative embodiment, the development code automatically triggers the testing process after submission, supporting automated regression testing.

[0089] Figure 5 This is a flowchart illustrating an integrated testing method as another exemplary embodiment of this application. The method is applied to an integrated testing framework based on a large AI model and the MCP protocol, including a model layer and an execution layer; such as... Figure 5 As shown, the method includes:

[0090] S501: Obtain the test requirements document for the target application, input the test requirements document into the large AI model located at the model layer, so that the large AI model performs the following operations under the guidance of the first model prompt:

[0091] S502: Perform multi-intent recognition on the test requirements document to obtain various test requirements; different test requirements include different test types and test objectives, and different test types are adapted to different test frameworks;

[0092] S503: Based on various testing requirements, including test types and test objectives, and combined with the target information of the target application, generate multiple test cases. The target information corresponding to different test types is different.

[0093] S504: Based on multiple test frameworks corresponding to various test types, convert multiple test cases into multiple test scripts that can be executed by the multiple test frameworks;

[0094] S505: Based on the MCP protocol, multiple test frameworks are called through multiple first MCP services located in the execution layer to execute multiple test scripts; among them, different first MCP services integrate calling interfaces of different test frameworks to call different test frameworks.

[0095] In this embodiment, the details of the integrated testing method can be found in the foregoing embodiments, and will not be repeated here.

[0096] In an optional embodiment, the integrated testing framework further includes: a scheduling layer located between the model layer and the execution layer, the scheduling layer including a target MCP service; when scheduling the multiple testing frameworks to execute the multiple test scripts based on the MCP protocol through multiple first MCP services located in the execution layer, the process includes: generating a first scheduling task instruction based on the multiple test types; inputting the first scheduling task instruction into the target MCP service, so that the target MCP service, guided by a second model prompt, selects multiple first MCP services adapted to the multiple test types from the multiple MCP services located in the execution layer according to the first scheduling task instruction, and returns the access links of the multiple first MCP services to the AI ​​large model; adding the access links of the multiple first MCP services to the MCP tool list of the AI ​​large model, and accessing the multiple first MCP services according to the access links of the multiple first MCP services and the multiple test scripts, so as to call the multiple testing frameworks to execute the multiple test scripts through the multiple first MCP services.

[0097] In one optional embodiment, the MCP service located in the execution layer is divided into multiple service domains, each service domain including MCP services providing different services. The multiple service domains include a first service domain, which includes multiple MCP services providing testing services by running different testing frameworks. Guided by the second model prompt, the target MCP service selects the first service domain from the multiple service domains according to the first scheduling task instruction, and selects multiple first MCP services from the multiple MCP services included in the first service domain. It then returns the identifier of the first service domain and the access links of the multiple first MCP services to the AI ​​big model. When adding access links for services to the MCP tool list of the AI ​​large model, and accessing the multiple first MCP services according to the access links of the multiple first MCP services and the multiple test scripts to call the multiple test frameworks to execute the multiple test scripts through the multiple first MCP services, the process includes: adding the identifier of the first service domain and the access links of the multiple first MCP services to the corresponding MCP tool list of the AI ​​large model; accessing the multiple first MCP services according to the identifier of the first service domain, the access links of the multiple first MCP services and the multiple test scripts to run the multiple test frameworks through the multiple first MCP services to execute the multiple test scripts.

[0098] In an optional embodiment, the plurality of service domains further includes: a second service domain and / or a third service domain, wherein the second service domain includes a second MCP service responsible for generating test cases, and the third service domain includes a third MCP service for converting test cases into test scripts; when generating multiple test cases based on the test types and test objectives included in the plurality of test requirements, combined with the target information of the target application, the process includes: generating a second scheduling task instruction based on the plurality of test types and the plurality of test objectives; inputting the second scheduling task instruction into the target MCP service, so that the target MCP service, guided by the second model prompt, selects a second service domain from the plurality of service domains according to the second scheduling task instruction, and returns the access link of the second MCP service in the second service domain to the AI ​​big model; adding the access link of the second MCP service to the MCP tool list of the AI ​​big model, and during the generation of the plurality of test cases, calling the second MCP service according to the access link of the second MCP service to assist in the generation of the plurality of test cases;

[0099] Accordingly, when converting the multiple test cases into multiple test scripts executable by the multiple test frameworks corresponding to the multiple test types, the process includes: generating a third scheduling task instruction based on the multiple test types; inputting the third scheduling task instruction into the target MCP service, so that the target MCP service, guided by the second model prompt, selects a third service domain from the multiple service domains according to the third scheduling task instruction, and returns the access link of the third MCP service in the third service domain to the AI ​​big model; adding the access link of the third MCP service to the MCP tool list of the AI ​​big model, and during the process of converting the multiple test cases into the multiple test scripts, calling the third MCP service according to the access link of the third MCP service to perform auxiliary conversion of the multiple test scripts.

[0100] In an optional embodiment, the method further includes: if it is identified that the target MCP does not return an access link to the first MCP service corresponding to the first test type, and the first test type is located within a set of test types, adding the access link of the target MCP service to the MCP tool list of the AI ​​large model; for the first test type, calling the target MCP service according to the access link of the target MCP service, so that the target MCP service can call a test framework adapted to the first test type to execute the test script corresponding to the first test type; the target MCP service integrates the calling interface of the test framework corresponding to each test type in the set of test types, and the first test type is any one of the multiple test types.

[0101] In one optional embodiment, the AI ​​large model is associated with a pre-built knowledge base; the knowledge base includes test annotation fragments corresponding to various test types, and each test annotation fragment corresponds to a different test requirement type tag; when performing multi-intent recognition on the test requirement document to obtain various test requirements, the process includes: performing semantic structure parsing on the test requirement document to obtain the hierarchical relationship between titles and paragraphs in the test requirement document; constructing a semantic structure tree based on the hierarchical relationship; determining candidate test requirement fragments for multi-intent recognition from the test requirement document based on the semantic structure tree, and expanding the context window of the candidate test requirement fragments; semantically matching the candidate test requirement fragments after expanding the context window with the test annotation fragments in the knowledge base to obtain a set of matching test requirement type tags; and obtaining the various test requirements based on the set of test type tags.

[0102] In an optional embodiment, before converting the test cases into test scripts, the method further includes: obtaining an application interface screenshot corresponding to the test requirement document; identifying interface elements in the application interface screenshot to determine the text and location information of interactive interface elements; constructing structured interface element knowledge entries based on the text and location information of the interactive interface elements and storing them in the knowledge base; correspondingly, when converting the multiple test cases into multiple test scripts executable by the multiple test frameworks corresponding to the multiple test types, the method includes: matching interface element knowledge entries from the knowledge base when the operation description of any test case involves an operation description of the application interface; and converting the multiple test cases into multiple test scripts executable by the multiple test frameworks based on the text and location information of the interactive interface elements in the matched interface element knowledge entries.

[0103] In one optional embodiment, the AI ​​large model includes multiple parallel channels, each channel having a corresponding test type label and configured with a test case template for the corresponding test type; based on the test types and test objectives included in the multiple test requirements, combined with the target information of the target application, multiple test cases are generated, including: matching the multiple test types with the test type labels corresponding to each channel to determine the channel suitable for each test requirement; generating the multiple test cases using the test case templates on the channel suitable for each test requirement and historical test cases; wherein, the multiple parallel channels are trained during the training of the AI ​​large model based on the results of classifying training samples, and new channels and their corresponding test type labels are generated based on new samples during the iteration process of the AI ​​large model.

[0104] In an optional embodiment, the above method further includes: generating an execution log during the execution of the test script; in the event of execution failure, returning the execution log to the AI ​​big model through the execution layer MCP service, so that the AI ​​big model can analyze the cause of the execution failure based on multiple dimensions of information, obtain the analysis results, generate repair suggestions based on the analysis results, and repair the information in the corresponding dimensions based on the repair suggestions.

[0105] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0106] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 501 to 503 can be device A; or the execution subject of steps 501 and 502 can be device A, and the execution subject of step 503 can be device B; and so on.

[0107] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 501, 502, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0108] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device includes a memory 64 and a processor 65.

[0109] Memory 64 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0110] Processor 65, coupled to memory 64, is used to execute computer programs in memory 64 for: acquiring a test requirement document for a target application; inputting the test requirement document into a large AI model located on the model layer, so that the large AI model, guided by a first model prompt, performs the following operations: performing multi-intent recognition on the test requirement document to obtain multiple test requirements; different test requirements include different test types and test objectives, and different test types are adapted to different test frameworks; generating multiple test cases based on the test types and test objectives included in the multiple test requirements, combined with the target information of the target application, with different target information corresponding to different test types; converting the multiple test cases into multiple test scripts that can be executed by the multiple test frameworks according to the multiple test types; and executing the multiple test scripts by calling the multiple test frameworks through multiple first MCP services located on the execution layer based on the MCP protocol; wherein, different first MCP services integrate calling interfaces for different test frameworks for calling different test frameworks.

[0111] In an optional embodiment, the integrated testing framework further includes a scheduling layer located between the model layer and the execution layer, the scheduling layer including a target MCP service; when the processor 65 schedules the multiple testing frameworks to execute the multiple test scripts based on the MCP protocol through multiple first MCP services located in the execution layer, it specifically performs the following: generating a first scheduling task instruction based on the multiple test types; inputting the first scheduling task instruction into the target MCP service, so that the target MCP service, guided by a second model prompt, selects multiple first MCP services adapted to the multiple test types from the multiple MCP services located in the execution layer according to the first scheduling task instruction, and returns the access links of the multiple first MCP services to the AI ​​large model; adding the access links of the multiple first MCP services to the MCP tool list of the AI ​​large model, accessing the multiple first MCP services according to the access links of the multiple first MCP services and the multiple test scripts, so as to call the multiple testing frameworks to execute the multiple test scripts through the multiple first MCP services.

[0112] In an optional embodiment, the MCP service located in the execution layer is divided into multiple service domains, each service domain including MCP services providing different services. The multiple service domains include a first service domain, which includes multiple MCP services providing testing services by running different testing frameworks. Guided by the second model prompt, the target MCP service selects the first service domain from the multiple service domains according to the first scheduling task instruction, and selects multiple first MCP services from the multiple MCP services included in the first service domain. It then returns the identifier of the first service domain and the access links of the multiple first MCP services to the AI ​​big model. The processor 65 then processes the multiple first MCP services... When adding the access link of the P service to the MCP tool list of the AI ​​large model, and accessing the multiple first MCP services according to the access links of the multiple first MCP services and the multiple test scripts, so as to call the multiple test frameworks to execute the multiple test scripts through the multiple first MCP services, specifically: adding the identifier of the first service domain and the access link of the multiple first MCP services to the corresponding MCP tool list of the AI ​​large model, and accessing the multiple first MCP services according to the identifier of the first service domain, the access link of the multiple first MCP services and the multiple test scripts, so as to run the multiple test frameworks through the multiple first MCP services to execute the multiple test scripts.

[0113] In an optional embodiment, the plurality of service domains further includes: a second service domain and / or a third service domain, wherein the second service domain includes a second MCP service responsible for generating test cases, and the third service domain includes a third MCP service for converting test cases into test scripts; when the processor 65 generates multiple test cases based on the test types and test objectives included in the plurality of test requirements, combined with the target information of the target application, it is specifically configured to: generate a second scheduling task instruction based on the plurality of test types and the plurality of test objectives; input the second scheduling task instruction into the target MCP service, so that the target MCP service, guided by the second model prompt, selects a second service domain from the plurality of service domains according to the second scheduling task instruction, and returns the access link of the second MCP service in the second service domain to the AI ​​big model; add the access link of the second MCP service to the MCP tool list of the AI ​​big model, and during the generation of the plurality of test cases, call the second MCP service according to the access link of the second MCP service to assist in the generation of the plurality of test cases;

[0114] Accordingly, when the processor 65 converts the multiple test cases into multiple test scripts executable by the multiple test frameworks corresponding to the multiple test types, it specifically performs the following: generating a third scheduling task instruction based on the multiple test types; inputting the third scheduling task instruction into the target MCP service, so that the target MCP service, guided by the second model prompt, selects a third service domain from the multiple service domains according to the third scheduling task instruction, and returns the access link of the third MCP service in the third service domain to the AI ​​big model; adding the access link of the third MCP service to the MCP tool list of the AI ​​big model, and during the process of converting the multiple test cases into the multiple test scripts, calling the third MCP service according to the access link of the third MCP service to perform auxiliary conversion of the multiple test scripts.

[0115] In an optional embodiment, the processor 65 is further configured to: if it is detected that the target MCP does not return an access link to the first MCP service corresponding to the first test type, and the first test type is located within a set of test types, add the access link of the target MCP service to the MCP tool list of the AI ​​large model; for the first test type, according to the access link of the target MCP service, call the target MCP service so that the target MCP service can call a test framework adapted to the first test type to execute the test script corresponding to the first test type; the target MCP service integrates the calling interface of the test framework corresponding to each test type in the set of test types, and the first test type is any one of the multiple test types.

[0116] In one optional embodiment, the AI ​​large model is associated with a pre-built knowledge base; the knowledge base includes test annotation fragments corresponding to various test types, and each test annotation fragment corresponds to a different test requirement type tag; when the processor 65 performs multi-intent recognition on the requirement document to obtain various test requirements, it is specifically used to: perform semantic structure parsing on the test requirement document to obtain the hierarchical relationship between the title and paragraph in the test requirement document; construct a semantic structure tree based on the hierarchical relationship; determine candidate test requirement fragments that need to be multi-intent recognition from the test requirement document based on the semantic structure tree, and expand the context window of the candidate test requirement fragments; perform semantic matching between the candidate test requirement fragments after expanding the context window and the test annotation fragments in the knowledge base to obtain a set of matching test requirement type tags; and obtain the various test requirements according to the set of test type tags.

[0117] In an optional embodiment, before converting the test cases into test scripts, the processor 65 is further configured to: obtain an application interface screenshot corresponding to the test requirement document; perform interface element recognition on the application interface screenshot to identify the text information and location information of the interactive interface elements; and construct structured interface element knowledge entries based on the text information and location information of the interactive interface elements and store them in the knowledge base.

[0118] Accordingly, when the processor 65 converts the multiple test cases into multiple test scripts that can be executed by the multiple test frameworks corresponding to the multiple test types, it includes: matching interface element knowledge entries from the knowledge base when the operation description of any test case involves an operation description of the application interface; and converting the multiple test cases into multiple test scripts that can be executed by the multiple test frameworks based on the text information and position information of interactive interface elements in the matched interface element knowledge entries.

[0119] In one optional embodiment, the AI ​​large model includes multiple parallel channels, each channel having a corresponding test type label and configured with a test case template for the corresponding test type; the processor 65 generates multiple test cases based on the test types and test objectives included in the multiple test requirements, combined with the target information of the target application, specifically for: matching the multiple test types with the test type labels corresponding to each channel to determine the channel suitable for each test requirement; generating the multiple test cases using the test case templates on the channel suitable for each test requirement and historical test cases; wherein, the multiple parallel channels are trained during the training of the AI ​​large model based on the results of classifying training samples, and new channels and their corresponding test type labels are generated based on new samples during the iteration process of the AI ​​large model.

[0120] In an optional embodiment, the processor 65 is further configured to: generate an execution log during the execution of the test script; and, in the event of execution failure, return the execution log to the AI ​​big model through the execution layer MCP service, so that the AI ​​big model can analyze the cause of the execution failure based on multiple dimensions of information, obtain the analysis results, generate repair suggestions based on the analysis results, and repair the information in the corresponding dimensions based on the repair suggestions.

[0121] Furthermore, such as Figure 6 As shown, the electronic device also includes other components such as a communication component 66, a display 67, a power supply component 68, and an audio component 69. Figure 6 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 6 The components shown. Additionally... Figure 6 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the form factor of the electronic device. The electronic device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server-side device such as a conventional server, cloud server, or server array. If the working node in this embodiment is implemented as a terminal device such as a desktop computer, laptop computer, or smartphone, it may include... Figure 6 The components within the dashed box; if the electronic device in this embodiment is implemented as a conventional server, cloud server, or server array, etc., it may be omitted. Figure 6 The component within the dashed box.

[0122] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0123] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0124] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0125] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0126] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0127] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium.

[0128] Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0129] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0130] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An integrated testing method, characterized by, The application is applied to an integrated test framework based on an AI large model and an MCP protocol, including a model layer and an execution layer; the method comprises: obtaining a test requirement document for a target application, inputting the test requirement document into an AI large model located on the model layer, so that the AI large model executes the following operations under the guidance of a first model prompt word: performing multi-intention recognition on the test requirement document to obtain multiple test requirements; different test requirements include different test types and test targets, and different test types adapt to different test frameworks; generating multiple test cases according to the test types and test targets included in the multiple test requirements, in combination with target information of the target application; the target information is different for different test types; converting the multiple test cases into multiple test scripts executable by the multiple test frameworks according to the multiple test frameworks corresponding to the multiple test types; calling the multiple test frameworks based on the MCP protocol through multiple first MCP services located on the execution layer to execute the multiple test scripts; different first MCP services integrate calling interfaces of different test frameworks for calling different test frameworks.

2. The method of claim 1, wherein, The integrated test framework further comprises a scheduling layer between the model layer and the execution layer, and the scheduling layer comprises a target MCP service; calling the multiple test frameworks based on the MCP protocol through multiple first MCP services located on the execution layer to execute the multiple test scripts, comprising: generating a first scheduling task instruction based on the multiple test types; inputting the first scheduling task instruction into the target MCP service, so that the target MCP service selects multiple first MCP services adapted to the multiple test types from multiple MCP services located on the execution layer according to the first scheduling task instruction under the guidance of a second model prompt word, and returns access links of the multiple first MCP services to the AI large model; adding the access links of the multiple first MCP services to an MCP tool list of the AI large model, and accessing the multiple first MCP services according to the access links of the multiple first MCP services and the multiple test scripts, so as to call the multiple test frameworks through the multiple first MCP services to execute the multiple test scripts.

3. The method of claim 2, wherein, The MCP services located on the execution layer are divided into multiple service domains, different service domains include MCP services providing different services, the multiple service domains include a first service domain, and the first service domain includes multiple MCP services providing test services by running different test frameworks; wherein, under the guidance of the second model prompt word, the target MCP service selects the first service domain from the multiple service domains according to the first scheduling task instruction, and selects the multiple first MCP services from the multiple MCP services included in the first service domain, and returns an identifier of the first service domain and access links of the multiple first MCP services to the AI large model; add the access links of the plurality of first MCP services to the MCP tool list of the AI large model, access the plurality of first MCP services according to the access links of the plurality of first MCP services and the plurality of test scripts, and call the plurality of test frameworks to execute the plurality of test scripts through the plurality of first MCP services, including: add the identification of the first service domain and the access links of the plurality of first MCP services to the MCP tool list of the AI large model, access the plurality of first MCP services according to the identification of the first service domain, the access links of the plurality of first MCP services and the plurality of test scripts, and run the plurality of test frameworks to execute the plurality of test scripts through the plurality of first MCP services.

4. The method of claim 3, wherein, The plurality of service domains further include a second service domain and / or a third service domain, the second service domain includes a second MCP service responsible for test case generation, and the third service domain includes a third MCP service for converting test cases into test scripts. According to the test types and test targets included in the plurality of test requirements, and in combination with the target information of the target application, a plurality of test cases are generated, including: According to the plurality of test types and the plurality of test targets, a second scheduling task instruction is generated; input the second scheduling task instruction into the target MCP service, so that the target MCP service selects a second service domain from the plurality of service domains according to the second scheduling task instruction under the guidance of a second model prompt word, and returns an access link of a second MCP service in the second service domain to the AI large model; add the access link of the second MCP service to the MCP tool list of the AI large model, and in the process of generating the plurality of test cases, call the second MCP service according to the access link of the second MCP service to assist in generating the plurality of test cases; According to the plurality of test frameworks corresponding to the plurality of test types, the plurality of test cases are converted into a plurality of test scripts executable by the plurality of test frameworks, including: generate a third scheduling task instruction based on the plurality of test types; input the third scheduling task instruction into the target MCP service, so that the target MCP service selects a third service domain from the plurality of service domains according to the third scheduling task instruction under the guidance of a second model prompt word, and returns an access link of a third MCP service in the third service domain to the AI large model; add the access link of the third MCP service to the MCP tool list of the AI large model, and in the process of converting the plurality of test cases into the plurality of test scripts, call the third MCP service according to the access link of the third MCP service to assist in converting the plurality of test scripts.

5. The method according to any one of claims 2-4, characterized in that, Further comprising: If it is detected that the target MCP does not return the access link of the first MCP service corresponding to the first test type, and the first test type is located within the set test type, the access link of the target MCP service is added to the MCP tool list of the AI ​​large model; For the first test type, based on the access link of the target MCP service, the target MCP service is invoked so that the target MCP service can call a test framework adapted to the first test type to execute the test script corresponding to the first test type; the target MCP service integrates the calling interface of the test framework corresponding to each test type in the set of test types, and the first test type is any one of the multiple test types.

6. The method of claim 1, wherein, The AI ​​model is associated with a pre-built knowledge base; the knowledge base includes test annotation fragments corresponding to various test types, and each test type corresponds to different test requirement type tags. The test requirement document is subjected to multi-intent recognition to obtain various test requirements, including: Perform semantic structure parsing on the test requirement document to obtain the hierarchical relationship between headings and paragraphs in the test requirement document; Based on the hierarchical relationship, a semantic structure tree is constructed; Based on the semantic structure tree, candidate test requirement fragments that need to be multi-intent recognition are determined from the test requirement document, and the context window of the candidate test requirement fragments is expanded. The candidate test requirement fragments after expanding the context window are semantically matched with the test annotation fragments in the knowledge base to obtain the set of test requirement type tags in the matching; Based on the set of test type tags, the various test requirements are obtained.

7. The method of claim 6, wherein, Before converting the test cases into test scripts, the following steps are also included: Obtain screenshots of the application interface corresponding to the aforementioned test requirements document; The screenshot of the application interface is used to identify interface elements, and the text information and location information of the interactive interface elements are identified. Based on the text and location information of the interactive interface elements, structured interface element knowledge entries are constructed and stored in the knowledge base. Accordingly, based on the multiple test frameworks corresponding to the various test types, the multiple test cases are converted into multiple test scripts that can be executed by the multiple test frameworks, including: In cases where the operation description of any test case involves an operation description of the application interface, the interface element knowledge entries matched from the knowledge base; Based on the text and location information of interactive interface elements in the matched interface element knowledge entries, the multiple test cases are converted into multiple test scripts that can be executed by the multiple test frameworks.

8. The method according to any one of claims 1 to 7, characterized in that, The AI ​​big model includes multiple parallel channels, each channel has a corresponding test type label and is configured with a test case template for the corresponding test type; Based on the test types and test objectives included in the various test requirements, and combined with the target information of the target application, multiple test cases are generated, including: The various test types are matched with the test type tags corresponding to each channel to determine the channel that matches each test requirement. The multiple test cases are generated using the test case templates on the channel adapted to each test requirement and historical test cases; The multiple parallel channels are obtained during the training of the AI ​​large model based on the results of classifying the training samples, and new channels and their corresponding test type labels are generated based on new samples during the iteration process of the AI ​​large model.

9. The method according to any one of claims 1 to 7, characterized in that, The method further includes: During the execution of the test script, an execution log is generated; In the event of execution failure, the execution log is returned to the AI ​​big model through the execution layer MCP service, so that the AI ​​big model can analyze the cause of the execution failure based on multiple dimensions of information, obtain the analysis results, generate repair suggestions based on the analysis results, and repair the information in the corresponding dimensions based on the repair suggestions; the test script is re-executed using the repaired information until the preset number of executions is reached, and then the manual interface is called for manual processing.

10. An electronic device, comprising: include: A memory and a processor, the memory for storing a computer program, the processor coupled to the memory for executing the computer program to implement the steps of the method of any one of claims 1-9.

11. A computer readable storage medium storing computer programs / instructions, characterized in that, When the computer program / instructions are executed by the processor, the processor is enabled to perform the steps of the method according to any one of claims 1-9.

Images