Test processing method and computer equipment

Abstract

The invention discloses a test processing method and computer equipment, and relates to the field of artificial intelligence and automatic testing, input information is received, the input information is natural language expression of a user for a test demand, the input information is processed based on a first model, and a test process description used for meeting the test demand is generated; the test process description comprises a plurality of structured test steps, then identifying a target skill required by the test steps through a first agent, and calling a target test script for realizing the target skill to execute a test operation based on a pre-established mapping relationship between the skill and the test script to obtain test execution data; wherein the first intelligent agent is configured to be an intelligent agent used for test execution and has a skill library containing a mapping relation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a test processing method and computer equipment. Background Technology

[0002] An automated testing platform is a software tool and framework that helps software development testing teams automate the testing process, such as automatically executing test cases, managing test data, and analyzing test results. This greatly reduces the workload of manual testing and improves testing efficiency, coverage, and accuracy.

[0003] However, current automated testing platforms tend to develop vertically, with testing functions often limited to a specific category. For example, testing platforms like Postman / Swagger focus on API (Application Programming Interface) testing and are typically used as API testing tools; while testing platforms like Selenium focus on UI testing and are typically used as web automation testing tools. This results in limited applicability of these testing tools, making it difficult to meet the diverse automated testing needs of the highly fragmented server testing field. Summary of the Invention

[0004] In view of the above problems, this application provides the following solution:

[0005] The first aspect of this application provides a test processing method, the method comprising:

[0006] Receive input information; the input information is the user's natural language expression of testing requirements;

[0007] The input information is processed based on the first model to generate a test process description that meets the test requirements; the test process description includes multiple structured test steps.

[0008] The first intelligent agent identifies the target skills required for the test steps, and based on the pre-established mapping relationship between skills and test scripts, calls the target test script that implements the target skills to execute test operations and obtain test execution data.

[0009] The first agent is configured as an agent for test execution and has a skill library containing the mapping relationship.

[0010] In one possible implementation, the first agent identifies the target skills required for the test steps, including:

[0011] The first intelligent agent matches the test steps with the skill description information in the skill library to obtain the corresponding skill matching degree.

[0012] Based on the skill matching degree, the target skills in the skill library are determined.

[0013] In one possible implementation, the process of establishing the mapping relationship includes:

[0014] Obtain the attribute characteristics of each test script in the test script library, as well as the preset skill library of the first agent; the skill library stores multiple predefined skills and skill description information for each skill;

[0015] Determine the degree of matching between the attribute features of each test script and the skill description information of each skill;

[0016] Based on the matching degree, a mapping relationship is established between the skill and at least one matching test script, and loaded into the skill library of the first agent.

[0017] In one possible implementation, the process of establishing the mapping relationship includes:

[0018] Obtain the attribute characteristics of each test script in the test script library;

[0019] Based on the second model, the attribute features of each test script are analyzed to generate structured skill description information; the skill description information corresponds to the attribute features of the corresponding test script.

[0020] The skill description information is registered to the first intelligent agent to construct the skill library of the first intelligent agent;

[0021] Based on the skill description information, determine the skills implemented by the corresponding test script;

[0022] Establish a mapping relationship between each skill and at least one test script that implements the skill, and load the skill library of the first agent.

[0023] In one possible implementation, the test operations are performed by a first intelligent agent, including:

[0024] The first intelligent agent injects the input data for the test steps into the target test script for execution.

[0025] The first intelligent agent triggers the third model to perform test operations on the input data through the model call interface in the target test script, so as to receive the test execution data of the test steps fed back by the third model.

[0026] In one possible implementation, the test execution data includes test process data and test result data, and the method further includes:

[0027] The second intelligent agent performs compliance checks on the test process data based on preset inspection dimensions.

[0028] The second intelligent agent performs a validity check on the test result data based on the test objectives or result verification rules of the test requirements.

[0029] The second agent is configured as an agent for testing and inspection;

[0030] The inspection dimensions include at least one of the following: script call standardization, input data injection accuracy, and test operation integrity.

[0031] The validity check includes at least one of the following: compliance of the test result format, consistency of the test result data with the test objective, and analysis of the reasonableness of abnormal test results.

[0032] In one possible implementation, the method further includes:

[0033] The third agent analyzes the inspection result data output by the second agent based on the output prompt information to generate the response information to the input information.

[0034] The output prompt information is determined based on any one of the input information, the context information of the input information, and the pre-configured output rules.

[0035] In one possible implementation, the first agent, in response to the test execution data, triggers the second agent to perform the step of checking the test execution data;

[0036] In response to the obtained inspection result data, the second agent triggers the third agent to generate the response information.

[0037] In one possible implementation, the step of analyzing the inspection result data output by the second agent based on the output prompt information by the third agent to generate the response information for the input information includes at least one of the following:

[0038] The third agent processes the inspection result data output by the second agent to generate a visual test report; the visual test report contains the inspection and analysis content of the test result data;

[0039] The third agent executes corresponding test response instructions based on the response levels corresponding to different inspection result data output by the second agent; the test response instructions include test stop instructions, test interruption instructions, or test execution instructions to execute the next test script.

[0040] A second aspect of this application provides a computer device, the computer device comprising:

[0041] A communication element is used to establish a communication connection between the computer device and the test platform, wherein multiple test scripts are stored on the test platform.

[0042] A processor, used to achieve the following based on at least one agent entering a working state:

[0043] Receive input information; the input information is the user's natural language expression of testing requirements;

[0044] The input information is processed based on the first model to generate a test process description that meets the test requirements; the test process description includes multiple structured test steps.

[0045] The first intelligent agent identifies the target skills required for the test steps, and based on the pre-established mapping relationship between skills and test scripts, calls the target test script that implements the target skills to execute test operations and obtain test execution data.

[0046] The first agent is configured as an agent for test execution and has a skill library containing the mapping relationship. Attached Figure Description

[0047] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0048] Figure 1 This is a flowchart illustrating a test processing method proposed in Embodiment 1 of this application;

[0049] Figure 2 This is a flowchart illustrating a test processing method proposed in Embodiment 2 of this application;

[0050] Figure 3 This is a flowchart illustrating a test processing method proposed in Embodiment 3 of this application;

[0051] Figure 4 This is a schematic flowchart of a test processing method proposed in Embodiment 4 of this application;

[0052] Figure 5 This is a flowchart illustrating a test processing method proposed in Embodiment 5 of this application;

[0053] Figure 6 This is a flowchart illustrating a test processing method proposed in Embodiment Six of this application;

[0054] Figure 7 This is a schematic diagram of the test processing device proposed in the embodiments of this application;

[0055] Figure 8 This is a schematic diagram of the hardware structure of a computer device proposed in an embodiment of this application. Detailed Implementation

[0056] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. The embodiments of this application are described below with reference to the accompanying drawings. It will be understood by those skilled in the art that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0057] The terms “first,” “second,” etc., used throughout this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0058] Reference Figure 1 The diagram below is a flowchart illustrating a test processing method proposed in Embodiment 1 of this application. This test processing method can be applied to computer devices, such as smartphones, laptops, desktop computers, or business terminals. This application does not impose any restrictions on these devices.

[0059] In this application, a smart program can run on the computer device to execute the test processing method of this application. This smart program can be built into the computer device's operating system or application program as part of the application or operating system, but is not limited to this deployment relationship. The smart program can be an intelligent agent (AI agent) or other program capable of providing intelligent functions in a conversational manner, autonomously perceiving the environment, analyzing data based on powerful model capabilities, and performing tasks to provide users with a natural and intelligent conversational interactive experience. It can also be an AI assistant built into the operating system, which can be activated and launched through voice or specific operations to implement the method provided in this application; this application does not limit its scope in this regard.

[0060] The aforementioned intelligent program has the ability to invoke various models (such as the first model, second model, and third model mentioned below in this application), such as through interface calls or other interactive methods. Different models can be invoked through corresponding different interfaces. Optionally, the aforementioned intelligent program may also have one or more models, that is, the model is part of the intelligent program. During the execution of the intelligent program by the processor, the intelligent program can launch one or more models to handle corresponding tasks according to actual needs. This application does not impose any restrictions on this.

[0061] Additionally, it should be noted that the model involved in this application can be a general AI (Artificial Intelligence) model, which can employ, but is not limited to, the Transformer or its architectural variants (such as using only an encoder-only / decoder-only architecture, an encoder-decoder architecture, or MoE (Mixture of Experts, a neural network architecture) or other basic architectures). It learns the features and patterns of natural language by training on large amounts of diverse data, thereby enabling it to understand and generate natural language. Typically, it has hundreds of millions to trillions of model parameters (model parameters are variables that control the behavior of the target model) and is capable of capturing complex relationships and patterns in natural language.

[0062] In this application, the AI ​​model may include, but is not limited to, generative models and generative language models (GLMs). For example, one or more of the following: large language model (LLM), GPT (Generative Pre-trained Transformer) series models, T5 (Text to Text Transfer Transformer) models, large visual models, and multimodal large models. Depending on actual needs, the models involved in the embodiments of this application may also be expert large models fine-tuned from general AI models based on application requirements, or lightweight models compressed through lightweight methods such as quantization, knowledge distillation, or pruning to meet the needs of edge deployments with limited computing resources. This application does not limit the model types of the models mentioned in the context and can be determined according to actual application requirements.

[0063] Based on the above analysis, such as Figure 1 As shown, the test processing method proposed in this embodiment may include:

[0064] Step S11: Receive input information; the input information is the user's natural language expression of the test requirements;

[0065] The input information can be a description of the test requirements obtained by the computer device in response to one or more input components, where the user (such as a tester with professional testing knowledge or an ordinary user without testing knowledge) performs a corresponding type of input operation. This description can also be a question or instruction format. Input components can include, but are not limited to, one or more of the following: an audio recorder, a touch screen, a finger, a stylus, a mouse and keyboard, or a joystick.

[0066] In practical applications, input information can be natural language input by the user in the form of voice to express the current testing requirements. In this case, the input information can be voice information input by the user through any input interface or state of the computer device; or it can be natural language input by the user in the form of video or image to represent the current testing requirements. In this case, the user can input image data through the real-time shooting interface of the computer device or other input interfaces.

[0067] Optionally, the input information can also be natural language expressing the current testing requirements in text form. For example, after waking up the agent, the current testing requirements can be entered in text form in the input box displayed by the agent. Of course, the input information can also be natural language in text form converted from the collected speech signal, or the content of the video or image can be recognized to obtain a natural language description of the current testing requirements. This application does not limit the form of the input information or the input implementation method.

[0068] Step S12: Based on the first model, process the input information to generate a test process description to meet the test requirements; the test process description contains multiple structured test steps.

[0069] The first model can be an AI model capable of semantic recognition and understanding of user intent (test requirements) from input information expressing natural language test requirements, planning the various test steps to implement the test tasks indicated by the test requirements, and generating a test process description containing these test steps and their execution order. Based on the above description of the model, this first model can be a generative model in the field of natural language processing, a multimodal / cross-modal generative model, or a pre-trained task-specific model, such as a model suitable for test scenarios based on a test platform, or a task-oriented generative model composed of multiple functional modules, such as a hybrid model based on rules and deep learning, used to generate structured test process descriptions such as test step instructions. This application does not limit the type of the first model.

[0070] Step S13: The first intelligent agent identifies the target skill required for the test step, and based on the pre-established mapping relationship between skills and test scripts, calls the target test script to implement the target skill to execute the test operation and obtain test execution data.

[0071] In this application, the first AI agent is configured as an agent for test execution, possessing a skill library containing the aforementioned mapping relationships. When a skill in the first AI agent's skill library needs to be activated, the first AI agent can automatically invoke one or more test scripts in the test platform to implement that skill based on the pre-established mapping relationships. This application does not limit the representation of these mapping relationships or their construction method. The skills possessed by the first AI agent can be atomic test skills, that is, test capabilities are decomposed into the smallest, indivisible test operation units. Each unit completes a specific, simple test task, which can be a test step or one of its smallest steps (sub-steps), such as clicking a button, entering text in an input box, or verifying whether a page element is displayed. If a test step contains multiple sub-steps, the identified target skill can include multiple corresponding atomic test skills, but is not limited to this.

[0072] For example, this application can establish mapping relationships through manually defined mapping rules (suitable for initial and small-scale scenarios, with strong controllability), mapping relationship models trained based on machine learning (suitable for large-scale scenarios, with a high degree of automation; also allowing optimization of initially manually established mapping relationships), and a hybrid approach of manual definition and machine learning (suitable for scenarios requiring a balance between accuracy and efficiency). This mapping relationship can also be dynamically adjusted based on feedback data from test results. It should be understood that, to distinguish different skills and different test scripts, a one-to-one corresponding identifier, such as an ID or other identifier, can be configured to establish a mapping relationship between different skill identifiers and different test script identifiers.

[0073] The first intelligent agent's skills refer to a set of reusable, goal-oriented "automated workflows" that can be model-driven, combined with external tools (such as testing tools in a testing platform) and data (such as input information and test scripts) to automatically complete specific complex tasks. Examples include: API (Application Programming Interface) interface testing tasks (such as interface function testing, performance benchmark testing, or data-driven testing); UI (User Interface) interface testing tasks (such as form validation testing, responsiveness and layout testing of test pages, or cross-application / device testing); performance and load testing tasks; security testing tasks (such as permission verification testing or vulnerability testing); compatibility testing tasks (such as mobile version testing, database compatibility testing, or cross-platform testing); accessibility testing tasks (such as scanning the entire homepage to check for accessibility barriers); or specialized testing tasks such as database, file operation / middleware (such as testing message queues).

[0074] Based on this, in one possible implementation, after the system determines that a test task needs to be executed and generates structured data (i.e., a test process description containing multiple test steps) describing the test task through the first model, it can automatically trigger the first agent to start. The test process description is input into the first agent, which sequentially identifies the test steps in the test process description and determines which one or more skills of the first agent need to be activated to execute the test step (which can be denoted as the target skill). This can be determined through information matching or the decision-making ability of the first agent itself, without any restrictions.

[0075] Therefore, the first intelligent agent in this application plays the role of a "super executor" and a "connector" between the user and the testing platform in the entire testing process. It seamlessly connects the high-level testing intent input by the user in natural language with the testing capabilities (test script library) of the testing platform. In the face of some difficulties in automated testing operations, such as image recognition and understanding, and configuration operations of the system / BIOS (Basic Input Output System) configuration interface, this application can conduct a comprehensive analysis of the testing steps based on the deep thinking ability and skills of the first intelligent agent, ensuring the reliability of the obtained test results. Testers do not need to manually select test scripts from the test platform's test script library based on their own knowledge, thus avoiding test failures caused by human selection errors. This achieves automated testing operations, improves testing efficiency and reliability, and allows testers to complete tests by directly communicating with the AI, greatly reducing the ability requirements for testers and improving the convenience and comprehensiveness of testing operations.

[0076] Reference Figure 2 This is a flowchart illustrating a test processing method proposed in Embodiment 2 of this application. This embodiment describes a possible implementation method for establishing the mapping relationship between the skills in the skill library of the first intelligent agent and the test script in the test processing method described above. Figure 2 As shown, the process of establishing this mapping relationship may include, but is not limited to:

[0077] Step S21: Obtain the attribute characteristics of each test script in the test script library, as well as the preset skill library of the first agent; the skill library stores multiple predefined skills and skill description information for each skill.

[0078] In this embodiment, when designing the first intelligent agent or extending the skills of an existing intelligent agent, a skill library for performing test operations supported by the first intelligent agent can be predefined based on actual needs. This library consists of a database of skills corresponding to different test operations or different types of test steps. For each skill, skill description information is configured, such as skill identifier, functional definition (a description of the skill's testing capabilities), input parameter requirements, output result format, and execution constraints. This clarifies what problems the corresponding skill of the first intelligent agent can solve, what needs to be provided and what is expected to be obtained in solving the problem, as well as the safety boundaries and operating environment of the skill. This allows the first intelligent agent to determine whether the skill needs to be activated and, after activating the skill, how to interact with the test platform and automatically perform test operations.

[0079] In this context, a skill identifier can be a unique capability index in the first agent's skill library, informing the agent's executor which skill to activate. The function definition of a skill describes its purpose and scope of application in a way that the first agent understands. For example, when it's necessary to perform a "login test," the first agent retrieves the function definitions of various skills to find those related to "login testing." The function definition can also serve as the basis for skill reuse and zero-shot learning, allowing the applicability of a skill to be determined by its match with the function definition when executing a new test step.

[0080] The input parameters of a skill specify the contextual information required for the first agent to execute the skill, i.e., what input parameters are necessary to invoke the skill. These parameters can be queried or generated from the input data of the test step to be executed or from the context of the test task to which it belongs, and are checked according to the definition to ensure that the parameters are complete and correctly formatted. The output structure format of a skill specifies the standardized information that the first agent promises to return after executing the skill. This information can serve as input for the next skill or as context for the next test step. The execution constraints of a skill specify the preconditions and safety boundaries for the successful activation of the skill. The skill will only be executed in a production environment that meets these conditions; otherwise, execution will be refused and an error reason will be returned. This endows the first agent with safety awareness and contextual awareness, improving the robustness and reliability of the AI ​​system.

[0081] Based on the above analysis, after developers extend the skills of the first intelligent agent, they can still define the skill description information of the extended skills according to the method described above, inject it into the first intelligent agent, and establish a mapping relationship between at least one test script for implementing the extended skill and the skill, according to the mapping relationship establishment method proposed in this application. Specifically, the predefined skills or extended skills can be determined based on historical test instances and development trends in the testing domain.

[0082] Furthermore, this application can parse the metadata information of each test script in the test script library of the testing platform, extracting the attribute characteristics of each test script, such as script identifier (e.g., script name), functional description (functional logic), input parameter specifications, execution constraints, and output data format, among others. The functional description of the test script can use natural language or concise statements to describe the purpose of the test script and the operations it performs, helping the first agent understand the purpose and applicable scenarios of the test script. The input parameter specifications of the test script can describe the name, data type, and necessity of each input parameter (e.g., input data for implementing test steps). The execution constraints of the test script describe the preconditions and limitations for executing the test script, such as the need for specific environment variables, specific test data preparation, required permissions, and dependent software or hardware environments. The output data format of the test script describes the data structure of the output result after the test script is executed, which may include the success or failure status of execution, as well as the possible returned data fields and types.

[0083] Step S22: Determine the degree of matching between the attribute features of each test script and the skill description information of each skill;

[0084] Step S23: Based on the matching degree, establish a mapping relationship between each skill and at least one matching test script, and load it into the skill library of the first agent.

[0085] In order to determine which skill of the first agent each test script can be used to implement, this application can perform matching analysis between the attribute features of each test script and the description information of each skill to obtain the matching degree between the corresponding test script and each skill. For the first number of K test scripts with the highest matching degree, or matching degree greater than the matching threshold, or matching degree higher after ranking, a mapping relationship is established with the skills, and they are loaded into the skill library of the first agent.

[0086] In the aforementioned matching analysis process, matching can be achieved through functional consistency (e.g., keyword matching by checking whether the nouns and functional descriptions of the test script contain skill keywords), parameter compatibility (including parameter matching and format consistency, such as the matching of test script parameters with skill requirements, and whether the test script output conforms to the expected skill format), and execution constraint adaptability. The matching scores of each matching method are then weighted and summed to obtain the matching degree between the corresponding test script and the skill. Optionally, this application can also convert the attribute features of the test script and the skill description information into vector forms, such as script feature vectors and skill description vectors. Then, the matching degree is determined by calculating the similarity between the script feature vectors and each skill description vector (e.g., vector distance, cosine similarity, or Euclidean distance calculation). Furthermore, this application can also analyze the matching degree between the attribute features of the test script and the skill description information through semantic similarity calculation. This application does not limit the implementation method of step S22.

[0087] After determining the matching degree between each test script and each predefined skill, the mapping relationship between each test script and the predefined skill can be determined through classification algorithms (pre-trained classifiers) / clustering methods. Optionally, the mapping relationship establishment method described above can be implemented by a machine learning model trained based on historical mapping data. In addition, after automatically determining the mapping relationship according to, but not limited to, the method described above, it can be fed back to relevant reviewers for manual review. In response to the review results provided by the reviewers, the final mapping relationship can be determined. There are no restrictions on the implementation method.

[0088] It should be noted that the established mapping relationship can be stored in a data format such as a graph structure or a list. It must at least contain the mapping relationship between the script identifier of the corresponding test script and the skill identifier of the corresponding skill. The relationship between the test script and the skill can be at least one of a one-to-one relationship, a many-to-one relationship, and a one-to-many relationship; this application does not impose any limitations on this. Preferably, after completing multiple test processes according to the method described in Embodiment 1 of this application, at least one quality evaluation indicator such as accuracy, recall, coverage, and diversity of the established mapping relationship can be evaluated based on historical test data and feedback data from testers on the mapping relationship or test results. The corresponding mapping relationship can then be adjusted accordingly, i.e., dynamic optimization of the mapping relationship can be achieved, improving the reliability of subsequent test processes.

[0089] In implementing the skill base mapping relationship of the first intelligent agent, a graph database can be used, but is not limited to, a graph structure for storing skills, test scripts, and mapping relationships. This supports skill-based script queries and skill-based reverse queries based on test scripts. It provides efficient mapping relationship retrieval (such as at least one of fuzzy retrieval based on skill keywords, semantic retrieval based on functional similarity, and intelligent retrieval based on execution features) and update interfaces. It can also maintain the version history and change records of mapping relationships, support batch import and backup / restore of mapping relationships, and provide statistical analysis and reporting functions for mapping relationships to meet actual testing and processing needs.

[0090] Reference Figure 3 This is a flowchart illustrating a test processing method proposed in Embodiment 3 of this application. This embodiment describes another possible method for establishing the mapping relationship between the skills in the skill library of the first intelligent agent and the test scripts in the test processing method described above. This method differs from the method described in Embodiment 2 above. Figure 3 As shown, the process of establishing the mapping relationship proposed in this embodiment may include, but is not limited to:

[0091] Step S31: Obtain the attribute characteristics of each test script in the test script library;

[0092] Step S32: Based on the second model, analyze the attribute features of each test script to generate structured skill description information; the skill description information corresponds to the attribute features of the corresponding test script.

[0093] In this embodiment, a test script-driven skill generation method can be adopted, that is, analyzing each test script in the test platform, introducing it as a skill of the first intelligent agent, and converting it into an execution instruction that the first intelligent agent can recognize. This achieves the standardization of non-standard original test scripts, so that the first intelligent agent can call different test cases through the execution instruction, automatically complete the corresponding test operations, and improve the script compatibility of the first intelligent agent.

[0094] In conjunction with the AI ​​model described above, the second model can be a fine-tuned large language model, a pre-trained information extraction model, or a specialized multimodal information extraction model. Depending on the specific needs, it can also be a rule-based or template-based processing engine, which can be determined based on the actual scenario. In this application, the second model is used to transform the attribute features of non-standardized or semi-structured test scripts into highly standardized skill description information that the first agent can understand and use. This application does not limit the implementation process of this transformation.

[0095] In one possible implementation, this application can pre-configure prompts representing the aforementioned transformation requirements, input these prompts along with the attribute features / metadata information of each test script into a second model. The second model extracts and understands each attribute feature to achieve standardized transformation and classification. After logical verification and inference and completion of missing information, it outputs the generated fully standardized skill description information. Thus, when a new test script is added to the testing platform, the second model can automatically generate skill description information for it and automatically register it with the first intelligent agent. This allows the first intelligent agent to automatically introduce the corresponding skills without manual intervention, thereby achieving automatic expansion and updating of the first intelligent agent's skill library.

[0096] Step S33: Register the skill description information to the first intelligent agent to build the skill library of the first intelligent agent;

[0097] Step S34: Based on the skill description information, determine the skills implemented by the corresponding test script;

[0098] Step S35: Establish a mapping relationship between each skill and at least one test script that implements the skill, and load the skill library of the first agent.

[0099] After generating skill description information for each skill according to, but not limited to, the method described above, in order to enable the first intelligent agent to possess the skill, the skill of the first intelligent agent can be directly activated during subsequent testing, the mapped test script can be called, and the corresponding test operation can be automatically executed. The skill description information can be injected into the first intelligent agent through the skill registration process, and a mapping relationship can be established between the corresponding skill and at least one test script that implements the skill (i.e., the test script used to generate the same or similar skill description information).

[0100] In the skill registration process, the skill registration manager can coordinate the entire process, including verifying skill description formats, verifying skill conflicts, storing skill metadata (skill description information, which is stored only after ensuring its completeness and correctness, such as through database or file system storage), building indexes (indexes based on at least one dimension, such as skill identifiers, keywords, or semantics, so that the first agent can identify target skills that match the testing requirements, and may also include mappings between the skills and test scripts, so that the first agent can call the target test scripts mapped to the target skills), and updating the first agent's runtime logic. Preferably, the first agent supports hot updates of skills without restarting it, thus avoiding disruption to the user's normal use of the first agent.

[0101] Reference Figure 4This is a flowchart illustrating a test processing method proposed in Embodiment 4 of this application. Based on the descriptions of the preceding embodiments, this embodiment can describe an optional detailed implementation process of the test processing method proposed in this application, such as... Figure 4 As shown, the detailed implementation process may include, but is not limited to:

[0102] Step S41: Receive input information; the input information is the user's natural language expression of the test requirements;

[0103] Step S42: Based on the first model, process the input information to generate a test process description to meet the test requirements; the test process description contains multiple structured test steps.

[0104] The implementation process of steps S41 and S42 can be referred to the description of the corresponding parts of the above embodiments, and will not be repeated in this embodiment.

[0105] Step S43: The first intelligent agent matches the test steps with the description information of each skill in the skill library to obtain the corresponding skill matching degree.

[0106] Step S44: The first intelligent agent determines the target skill in the skill library based on the skill matching degree;

[0107] Based on the above description of the mapping relationship establishment process, since the first intelligent agent already possesses a skill library, and each skill has structured skill description information, this application can, through matching methods described above, analyze the intent and requirements of the test step and perform semantic analysis with the skill description information of each skill to obtain the corresponding skill matching degree. For example, the matching degree can be obtained by converting the result into a vector and then performing similarity calculation, or by calculating semantic similarity. Alternatively, the skill matching degree between the test step and each skill can be determined by matching the keywords of the test step with the skill description information. Then, the skill with the highest matching degree (or the skill with a matching degree greater than a threshold) is selected as the target skill required to implement the test step. Furthermore, the context information of the test task to which the test step belongs can be incorporated into the above matching process. This application does not restrict the implementation process of steps S43 and S42.

[0108] Step S45: The first agent calls the target test script to achieve the target skill based on the pre-established mapping relationship between skills and test scripts.

[0109] Step S46: The first agent injects the input data of the test steps into the target test script for execution;

[0110] Based on the above analysis, after the first intelligent agent determines which skill (target skill) needs to be activated to implement the current test step to be executed, it queries the stored mapping relationships based on the skill identifier of the target skill to determine the script identifier corresponding to the skill identifier, so as to call the target test script with the script identifier (which may be one or more test scripts already existing on the test platform), such as by sending an execution instruction containing the script identifier to the test platform to control the test platform to execute the target test script.

[0111] In the process of executing the target test script as described above, the first intelligent agent can synchronize or send the input data acquired to implement the test steps to the test platform when needed, so as to inject the corresponding fields of the target test script and implement the test steps. This application does not describe the implementation process in detail.

[0112] In step S47, the first agent triggers the third model to perform test operations on the input data through the model call interface in the target test script, so as to receive the test execution data of the test steps fed back by the third model.

[0113] In some embodiments, for some difficult operations in traditional automated testing, such as image recognition and understanding, and BIOS / system configuration interface operations, the first agent can call the corresponding model to implement the test operation during the execution of the target test script. The target test script may include the corresponding model call interface. During the execution of the target test script, the first agent detects the model call interface and can directly call the third model to execute the test operation using the input data of the current test step.

[0114] For example, the image data to be processed in the image recognition step is input into a third model for image recognition, and the image recognition result is output, triggering the next test step or other target test scripts in the current test step; or, based on the display information and configuration tasks of the BIOS configuration interface required for the operation steps of the BIOS configuration interface, a configuration prompt word is constructed, and the configuration prompt word is input into the third model to guide the third model to reason and generate a set of configuration actions for the BIOS configuration interface, so that the first intelligent agent executes the configuration actions sequentially and automatically completes the operation of the BIOS configuration interface. However, it is not limited to the two test scenarios listed in this application.

[0115] In the process of the first intelligent agent calling the target test script to execute the test operation, the first intelligent agent can record the test process data and test result data of each test step in the entire test process as test execution data storage for subsequent query or inspection, etc. This application does not restrict the content and storage method of the test execution data.

[0116] Reference Figure 5This is a flowchart illustrating a test processing method proposed in Embodiment 5 of this application. Based on the test processing methods described in the preceding embodiments, this application can further utilize a first intelligent agent and a second intelligent agent to collaboratively check the test execution data. If necessary, it can also collaborate with a third intelligent agent to generate feedback information such as test reports for the user. This application does not impose any limitations on this. Figure 5 As shown, after the first agent obtains the test execution data, the test processing method proposed in this embodiment can also perform, but is not limited to, the following steps:

[0117] Step S51: The second intelligent agent performs a compliance check on the test process data obtained by the first intelligent agent based on a preset inspection dimension.

[0118] Step S52: The second agent checks the validity of the test result data obtained by the first agent based on the test objectives or result verification rules of the test requirements.

[0119] The second agent is configured as a test and inspection agent, which is an independent agent distinct from the first agent. It is used to automatically test and inspect the test execution data generated during the test operations performed by the first agent, such as test process data and test result data. It can be seen that this test and inspection capability can be used as a proprietary skill of the second agent. By utilizing its own deep thinking ability, decision-making ability or model invocation ability, it can automatically realize test and inspection. Compared with manual inspection, it reduces the cost of manual testing and the ability requirements of testers. Testers do not need to know the correct execution data of each test operation, but can also know the inspection results of the test operation, thus improving the accuracy and comprehensiveness of test and inspection.

[0120] Based on this, during the test execution process implemented by the first intelligent agent as described in the above embodiments, or after the test execution is completed, the second intelligent agent can be used to check the test execution data to determine whether the test operation implemented by the first intelligent agent is correct and whether the test result is correct. If there are any abnormalities, the corresponding abnormal data can be recorded to adjust the test process or optimize the test script, etc.

[0121] Reference Figure 6The flowchart shown illustrates that this application can implement the proposed test processing method through multi-agent collaboration. In practical applications, the second agent can monitor the execution of the target test script by the first agent, actively acquiring and checking the generated test execution data. Alternatively, the first agent can respond to the test execution data and trigger the second agent to perform the step of checking the test execution data. Optionally, the system can monitor the execution process of the first agent, and after obtaining the test execution data, analyze it to determine if it needs to be checked, and then trigger the second agent to perform the aforementioned checking steps. This application does not restrict the triggering method of the second agent.

[0122] The pre-configured inspection dimensions in the second intelligent agent may include, but are not limited to, at least one of the following: script call standardization, input data injection accuracy (e.g., whether the injected content and format of each field in the target test script are correct, which can be achieved based on the constraints or format requirements of the target test script), and test operation integrity (which can be determined based on the description of the test steps or the expected test objectives). The above-mentioned validity checks on test result data may include, but are not limited to, at least one of the following: test result format compliance, consistency between test result data and test objectives, and analysis of the rationality of abnormal test results. The implementation process of each check is not detailed in this application. It should be understood that, at any time, updates to these inspection rules can synchronously update the above-mentioned inspection steps of the second intelligent agent, or update the inspection prompts used to describe the above-mentioned inspection rules, so that the second intelligent agent can perform checks on the test execution data based on the latest inspection prompts, but is not limited to this.

[0123] Step S53: The third agent analyzes the inspection result data output by the second agent based on the output prompt information to generate a response message for the input information; the output prompt information is determined based on any one of the input information, the context information of the input information, and the pre-configured output rules.

[0124] In cases where it is necessary to provide users with response information in a specific format based on the test result data, this application may also call a third agent to generate response information based on the inspection result data generated by the second agent, such as a visualized test report or other format response information. As needed, the response information may also include analysis of the test results, especially analysis of test anomalies, in order to maintain the skill library and test script library, etc.

[0125] In one possible implementation, the second agent, in response to the obtained inspection result data, triggers the third agent to generate response information, such as... Figure 6As shown, the second agent sends the obtained inspection result data to the third agent to generate a response message. Optionally, the third agent can also monitor the second agent's inspection process of the test execution data, read the test result data generated by the third agent, and generate a response message based on the determined output prompt information. This application does not limit the triggering implementation process of the third agent, including but not limited to... Figure 6 The method proposed in this application is implemented by sequentially calling the first, second, and third intelligent agents. Optionally, for these three intelligent agents, the steps they implement can be divided into multiple sub-steps as needed, which can be implemented by calling the corresponding sub-intelligent agents. In this case, each sub-intelligent agent can be an independent intelligent agent with the processing capability of the corresponding sub-step, and can implement the sub-step based on its own decision-making ability or deep thinking ability, or by calling a suitable model. This application does not impose any restrictions on this. The calling and implementation process is similar to... Figure 6 The implementation process of multi-agent cooperation shown is similar and will not be described in detail in this application.

[0126] The output prompt information can describe the processing rules for the inspection result data of the input third agent, the format (pre-configured output rules) for generating the response information of the input information, etc. This application does not limit the method for determining the output prompt information and its content, and it can be dynamically adjusted as the information it depends on is updated, so as to dynamically update the output form and content of the response information, etc. The implementation process is not described in detail in this application.

[0127] In some embodiments, this application can process the inspection result data output by the second agent using a third agent to generate a visual test report (i.e., a response message, but not limited to this). This visual test report includes the inspection and analysis content of the test result data, such as the reasons for any test anomalies. Optionally, the third agent can also execute corresponding test response instructions based on the response levels corresponding to different inspection result data output by the second agent (e.g., determined based on the severity of the test anomalies) to intervene in or control the testing process. These test response instructions include stop test instructions, interrupt test instructions, or test execution instructions to execute the next test script.

[0128] In this application, a stop test command is equivalent to halting the entire test process, preventing the execution of any subsequent test cases / scripts. This typically occurs when a very serious error occurs, rendering subsequent testing impossible or meaningless, such as a test environment crash, inaccessibility of the system under test, failure of preconditions, or failure to achieve the test objective. The third-party agent can automatically identify and execute the stop test command to control the test process. An interrupt test command may be similar to a stop test command, but may emphasize abnormal termination and may be recorded in a different state. In the event of an unrecoverable error or a serious system failure, such as a server crash, security vulnerabilities, exhaustion of system resources, or external emergencies requiring immediate release of the test environment due to power outages or hardware failures, the third-party agent identifies and executes the interrupt test command, triggering emergency cleanup operations before exiting the test.

[0129] In some embodiments, this application can execute the next test script when the response level is either pass or warning, such as when the current tests are all normal, a non-blocking defect is found but does not affect subsequent tests, or the test results all meet the quality requirements for continuing execution. This can be controlled according to actual needs. If the response level is failure, a stop test command can be executed; if the response level is error, an interrupt test command can be executed, without restriction. Depending on actual needs, especially in more complex test scenarios, other factors, such as the importance of test steps and test stages / progress, can be considered to determine the response level. This allows the third-party intelligent agent to execute appropriate test response commands based on its own decision-making capabilities, thereby quickly responding to serious problems, avoiding resource waste, ensuring test environment security, preventing problem spread, optimizing test execution efficiency, intelligently adjusting test strategies, and providing a clear test status for easy problem tracking and decision-making.

[0130] Reference Figure 7 This is a schematic diagram of the test processing device proposed in an embodiment of this application. This test processing device can be applied to computer equipment, such as... Figure 7 As shown, the test processing apparatus proposed in this embodiment may include:

[0131] The input information receiving module 71 is used to receive input information; the input information is the user's natural language expression of testing requirements.

[0132] The test process description generation module 72 is used to process the input information based on the first model and generate a test process description to meet the test requirements; the test process description includes multiple structured test steps.

[0133] The test execution module 73 is used to identify the target skill required for the test step through the first intelligent agent, and based on the pre-established mapping relationship between skills and test scripts, call the target test script that implements the target skill to execute the test operation and obtain test execution data;

[0134] The first agent is configured as an agent for test execution and has a skill library containing the mapping relationship.

[0135] In some embodiments, the test processing apparatus may further include a mapping relationship establishment module for establishing mapping relationships between different skills and test scripts in the test platform; the mapping relationship establishment module may include:

[0136] The first acquisition unit is used to acquire the attribute characteristics of each test script in the test script library, and the preset skill library of the first agent; the skill library stores multiple predefined skills and skill description information for each skill.

[0137] The first matching unit is used to determine the degree of matching between the attribute features of each test script and the skill description information of each skill;

[0138] The first establishment unit is used to establish a mapping relationship between the skill and at least one matching test script based on the matching degree, and load it into the skill library of the first agent.

[0139] Optionally, the mapping relationship establishment module mentioned above may also include:

[0140] The second acquisition unit is used to acquire the attribute characteristics of each test script in the test script library;

[0141] The first generation unit is used to parse the attribute features of each test script based on the second model and generate structured skill description information; the skill description information corresponds to the attribute features of the corresponding test script.

[0142] A registration unit is used to register the skill description information to the first intelligent agent in order to build the skill library of the first intelligent agent;

[0143] The first determining unit is used to determine the skill implemented by the corresponding test script based on the skill description information;

[0144] The second establishment unit is used to establish a mapping relationship between each skill and at least one test script that implements the skill, and to load the skill library of the first agent.

[0145] In some embodiments, the test execution module 73 described above may include:

[0146] The second matching unit is used to match the test steps with the skill description information in the skill library through the first intelligent agent to obtain the corresponding skill matching degree.

[0147] The second determining unit is used to determine the target skill in the skill library based on the skill matching degree.

[0148] Optionally, the test execution module 73 may further include:

[0149] An injection unit is used to inject the input data for implementing the test steps into the target test script for execution through a first intelligent agent;

[0150] An execution unit is used by the first intelligent agent to trigger the third model to perform test operations on the input data through the model call interface in the target test script, so as to receive the test execution data of the test steps fed back by the third model.

[0151] The test execution data may include test process data and test result data. Based on this, the aforementioned test processing device may further include:

[0152] The first inspection module is used to perform compliance checks on the test process data based on preset inspection dimensions by a second intelligent agent; the second intelligent agent is configured as an intelligent agent for test inspection.

[0153] The second inspection module is used to perform validity checks on the test result data based on the test objectives or result verification rules of the test requirements by the second intelligent agent.

[0154] The inspection dimensions include at least one of script call standardization, input data injection accuracy, and test operation integrity; the validity inspection includes at least one of test result format compliance, test result data consistency with the test objective, and abnormal test result rationality analysis.

[0155] Based on the above analysis, the above-mentioned test processing device may further include:

[0156] The response information generation unit is used to analyze the inspection result data output by the second agent based on the output prompt information by the third agent, and generate response information for the input information; wherein the output prompt information is determined based on any one of the input information, the context information of the input information, and the pre-configured output rules.

[0157] Specifically, the aforementioned inspection module is used to trigger the second agent to inspect the test execution data in response to the test execution data; the response information generation unit is used to trigger the third agent to generate the response information in response to the obtained inspection result data.

[0158] Optionally, the response information generation unit may include at least one of the following sub-units:

[0159] A generation subunit is used to process the inspection result data output by the second agent through a third agent and generate a visual test report; the visual test report contains the inspection and analysis content of the test result data;

[0160] The execution subunit is used to execute corresponding test response instructions by a third agent based on the response level corresponding to different inspection result data output by the second agent; the test response instructions include a stop test instruction, an interrupt test instruction, or a test execution instruction to execute the next test script.

[0161] This application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on a computer device, the computer device implements any of the test processing methods provided in this application. The computer program product can be stored in a readable storage medium, such as a computer floppy disk, USB flash drive, portable hard drive, ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training device, or network device, etc.) to execute the test processing methods described in the various embodiments of this application.

[0162] This application also provides a computer-readable storage medium carrying one or more computer programs. When these programs are executed by a computer device, the computer device can implement any of the test processing methods provided in this application. This application does not limit the product form of the computer-readable storage medium.

[0163] Reference Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application. The computer device may include a communication element 81 and a processor 82, wherein:

[0164] The communication element 81 can be used to realize the communication connection between the computer device and the test platform (server), which stores multiple test scripts. The processor 82 can enter the working state based on at least one intelligent agent to implement the various steps of the test processing method proposed in the embodiments of this application. The implementation process can be referred to the description of the corresponding part of the method embodiments above, and will not be repeated here.

[0165] Furthermore, the computer device may also include at least one memory 83, which can be used to store programs / instructions required to execute the test processing method, so that the processor executes the programs / instructions to implement the test processing method proposed in the embodiments of this application. Optionally, the memory 83 may also store at least one of the models deployed on the edge, such as the first model, the second model, and the third model mentioned above.

[0166] In addition, the computer device may also include at least one input component 84, such as an audio acquisition device, stylus, mouse and keyboard, or joystick, for users to operate the input component 84 to input input information representing test requirements into the computer device. This application does not describe in detail the type of input component 84 or its input principle. The computer device may also include at least one output component 85, such as an audio player or display screen, for outputting response information to the input information, such as a visual test report, etc., without limitation.

[0167] It should be understood that, Figure 8 The structure of the computer device shown does not constitute a limitation on the computer device in the embodiments of this application. In practical applications, the computer device may include more than Figure 8 The application does not provide detailed examples of the more or fewer components shown, or combinations of certain components, such as gyroscopes, accelerometers, and gravity sensors used to obtain sensing parameters, power management modules, antennas, or other communication elements.

[0168] Finally, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0169] In the above embodiments, the invention can be implemented entirely or partially by software, hardware, firmware, or any combination thereof. Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware, or it can be implemented using dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memory, dedicated components, etc. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The various embodiments in this specification are described in a progressive or parallel manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to mutually. For the apparatus and computer equipment disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

Claims

1. A test processing method, the method comprising: Receive input information; The input information is the user's natural language expression of their testing requirements; Based on the first model, the input information is processed to generate a test process description to meet the test requirements; The test process description includes multiple structured test steps; The first intelligent agent identifies the target skills required for the test steps, and based on the pre-established mapping relationship between skills and test scripts, calls the target test script that implements the target skills to execute test operations and obtain test execution data. The first agent is configured as an agent for test execution and has a skill library containing the mapping relationship.

2. The method according to claim 1, wherein, The first intelligent agent identifies the target skills required for the test steps, including: The first intelligent agent matches the test steps with the skill description information in the skill library to obtain the corresponding skill matching degree. Based on the skill matching degree, the target skills in the skill library are determined.

3. The method according to claim 1, wherein, The process of establishing the mapping relationship includes: Obtain the attribute characteristics of each test script in the test script library, as well as the preset skill library of the first agent; the skill library stores multiple predefined skills and skill description information for each skill; Determine the degree of matching between the attribute features of each test script and the skill description information of each skill; Based on the matching degree, a mapping relationship is established between the skill and at least one matching test script, and loaded into the skill library of the first agent.

4. The method according to claim 1, wherein, The process of establishing the mapping relationship includes: Obtain the attribute characteristics of each test script in the test script library; Based on the second model, the attribute features of each test script are analyzed to generate structured skill description information; the skill description information corresponds to the attribute features of the corresponding test script. The skill description information is registered to the first intelligent agent to construct the skill library of the first intelligent agent; Based on the skill description information, determine the skills implemented by the corresponding test script; Establish a mapping relationship between each skill and at least one test script that implements the skill, and load the skill library of the first agent.

5. The method according to any one of claims 1-4, wherein, The test operations are performed by the first intelligent agent, including: The first intelligent agent injects the input data for the test steps into the target test script for execution. The first intelligent agent triggers the third model to perform test operations on the input data through the model call interface in the target test script, so as to receive the test execution data of the test steps fed back by the third model.

6. The method according to any one of claims 1-4, wherein the test execution data includes test process data and test result data, and the method further includes: The second intelligent agent performs compliance checks on the test process data based on preset inspection dimensions. The second intelligent agent performs a validity check on the test result data based on the test objectives or result verification rules of the test requirements. The second agent is configured as an agent for testing and inspection; The inspection dimensions include at least one of the following: script call standardization, input data injection accuracy, and test operation integrity. The validity check includes at least one of the following: compliance of the test result format, consistency of the test result data with the test objective, and analysis of the reasonableness of abnormal test results.

7. The method according to claim 6, further comprising: The third agent analyzes the inspection result data output by the second agent based on the output prompt information to generate the response information to the input information. The output prompt information is determined based on any one of the input information, the context information of the input information, and the pre-configured output rules.

8. The method according to claim 7, wherein: In response to the test execution data, the first agent triggers the second agent to perform the step of checking the test execution data; In response to the obtained inspection result data, the second agent triggers the third agent to generate the response information.

9. The method according to claim 7, wherein the step of analyzing the inspection result data output by the second intelligent agent based on the output prompt information by the third intelligent agent to generate the response information to the input information includes at least one of the following: The third agent processes the inspection result data output by the second agent to generate a visual test report; the visual test report contains the inspection and analysis content of the test result data; The third agent executes corresponding test response instructions based on the response levels corresponding to different inspection result data output by the second agent; the test response instructions include test stop instructions, test interruption instructions, or test execution instructions to execute the next test script.

10. A computer device, the computer device comprising: A communication element is used to establish a communication connection between the computer device and the test platform, wherein multiple test scripts are stored on the test platform. A processor, used to achieve the following based on at least one agent entering a working state: Receive input information; the input information is the user's natural language expression of testing requirements; Based on the first model, the input information is processed to generate a test process description to meet the test requirements; The test process description includes multiple structured test steps; The first intelligent agent identifies the target skills required for the test steps, and based on the pre-established mapping relationship between skills and test scripts, calls the target test script that implements the target skills to execute test operations and obtain test execution data. The first agent is configured as an agent for test execution and has a skill library containing the mapping relationship.

Images