Test method and device, medium, equipment and computer program product

Abstract

A test method, apparatus, medium, device and computer program product, the method comprising: receiving a description text of a test case, the description text being a text written based on a natural language, the test case comprising at least one test step; based on the description text and a first model, an execution agent and execution parameters corresponding to the test step are determined from multiple test agents, and the test agents are used for calling execution interfaces corresponding to the test agents; and determining an execution result of the test step based on the execution agent, the execution parameter and the execution interface. Therefore, a user can compile the test case in a natural language form, a large amount of human input for understanding and coding is not needed, the description text is analyzed through the model to call the test agent, the manual workload of testers is effectively reduced, and the test efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a testing method, apparatus, medium, device, and computer program product. Background Art

[0002] In backend microservice systems, system links and business logic are primarily derived through data processing, data storage, data communication, and model-based decision-making. The processing results of these microservices are typically fed back to the service caller for subsequent processing. To ensure the effective operation of microservices, functional performance testing is often required to confirm that the services can achieve their intended functions.

[0003] In related technologies, the execution process or results of microservices are usually not displayed interactively and visually. How to effectively implement microservice testing is an urgent problem that test development engineers need to solve. Summary of the Invention

[0004] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a testing method, the method comprising: receiving a description text of a test case, wherein the description text is a text written based on a natural language, and the test case includes at least one test step; Determining, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents, wherein the test agent is used to call an execution interface corresponding to the test agent; An execution result of the test step is determined based on the execution agent, the execution parameters, and the execution interface.

[0006] In a second aspect, the present disclosure provides a testing device, comprising: A receiving module, configured to receive a description text of a test case, wherein the description text is a text written based on a natural language, and the test case includes at least one test step; a first determining module configured to determine, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents, wherein the test agent is configured to call an execution interface corresponding to the test agent; The first testing module is configured to determine an execution result of the test step based on the execution agent, the execution parameters, and the execution interface.

[0007] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processing device.

[0008] In a fourth aspect, the present disclosure provides an electronic device, comprising: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0009] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0010] Through the above technical solution, users can write test cases in natural language, which allows the model to identify the test agent used to execute the test steps. The test agent then calls the corresponding execution interface to implement the test process of the test steps. This process eliminates the need for extensive human effort in understanding and coding. The model analyzes the descriptive text to call the test agent, allowing batch execution of natural language test cases, effectively reducing the manual workload of testers. Furthermore, testers only need to write natural language test cases, eliminating the need for different users to write test code in different styles. This reduces the complexity of subsequent maintenance and test attribution analysis, lowers the technical requirements for testers, and improves testing efficiency.

[0011] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flowchart of a testing method provided according to an embodiment of the present disclosure.

[0013] Figure 2 It is an interactive schematic diagram of a testing method provided according to an embodiment of the present disclosure.

[0014] Figure 3 It is a schematic diagram of a display interface provided according to an embodiment of the present disclosure.

[0015] Figure 4 is a block diagram of a testing device provided according to one embodiment of the present disclosure.

[0016] Figure 5 A schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0017] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0018] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0019] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0024] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0025] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0026] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0027] At the same time, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0028] Figure 1 As shown in FIG, it is a flow chart of a testing method provided according to an embodiment of the present disclosure. Figure 1 As shown, the method may include: In step 11, a description text of a test case is received, wherein the description text is a text written based on a natural language, and the test case includes at least one test step.

[0029] The test case is a test case for testing the microservice system. In this step, the test user can write the processing logic of the test case in natural language. For example, the test case is represented as follows: Test case name: Creative review, single creative callback, exceptions found in multiple regions.

[0030] 1. Create a C-type creative content in areas A1 and A2. If the conditions of T1 and T2 are met, the creative content will be manually reviewed. 2. Wait 2 minutes; 3. Wait for 8 minutes; 4. Query the audit log based on the creative ID and status ID, find n records, and consider the creative content to enter the manual review module; 5. Send queue message, the message format is {XXX1}; 6. Wait 2 minutes; 7. Query the audit log based on the creative ID and status ID. 2 records are found. Verify that the policy in the data contains ID X. 8. Query the audit log based on the creative identifier and status identifier, and find two records. Verify that the policy in the data contains identifier X.

[0031] As shown in the example above, the test user can describe the test case based on natural language to describe the expected results of the microservice system during operation. Among them, the test steps can be detected in advance. If it is determined that the test step is a preset operation, the test step can be executed directly without determining the test agent. For example, for test steps 2, 3, and 6, the test step execution completion can be determined after waiting for a corresponding time. It is used to indicate the time required to run the corresponding processing logic in the microservice system, and the subsequent steps are performed after the corresponding processing logic is executed by the microservice system. If the test step is not a preset operation, such as steps 1, 4, 5, 7, and 8 above, its corresponding execution agent and execution parameters can be determined based on the first model to execute the test step.

[0032] In step 12, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step are determined from a plurality of test agents, wherein the test agent is used to call an execution interface corresponding to the test agent.

[0033] In this embodiment, multiple interfaces can be pre-set based on the operations required to be performed during the test process, so that a test agent can be further constructed to implement calls to the corresponding interfaces based on the agent. For example, multiple test agents can be constructed based on the test scenario, and the test agent can include multiple of the following: a data construction agent for data construction, a database query agent for querying data from a database, a message sending agent for message interaction, a log query agent for querying audit logs, a log query agent for querying Argos logs, a code agent for automatically generating code based on test cases, and a generation agent for generating test reports, etc.

[0034] As an example, a corresponding execution interface can be configured based on the actual application scenario, a test agent corresponding to the execution interface can be configured, and the test agent can be registered. The test agent registration process can be implemented based on agent registration methods commonly used in the art, and this disclosure is not limited to this. After registering the test agent, a description of the test agent can be obtained, which indicates the test functions implemented by the test agent.

[0035] The first model can be implemented based on a Large Language Model (LLM). For example, in this step, the input text of the first model can be constructed based on the description text and the description information of the test agent, according to the prompt text of the first model. This input text can then be input into the first model, which then analyzes the input text to determine which execution agent to use to execute the test step, thereby obtaining an execution agent and execution parameters. The execution parameters are parameters corresponding to the execution interface corresponding to the execution agent, and are used to implement the call of the execution interface.

[0036] In step 13, the execution result of the test step is determined based on the execution agent, the execution parameters and the execution interface.

[0037] In this step, a request body for calling the execution interface can be constructed based on the execution agent and execution parameters. The execution interface is then called based on the request body to obtain a processing result of the execution interface. The execution result is then determined based on the processing result and the step text of the test step in the description text. The execution result can include the processing result and whether the test step was successful.

[0038] As an example, for step 1 in the above test case, the request body can be constructed using the execution agent and execution parameters corresponding to step 1. The execution agent is the data construction agent, and the parameters of the constructed request body are expressed as follows: 1. b_line: set to 1, indicating creative content under type C; 2. V_list: Contains two creative contents, uses approved materials and is refreshed; 3. r_list: set to A1 and A2, indicating that the regions of the creative content are A1 and A2; 4. ad_type: Set to 3, the default value for the task type of creative content; 5. human_mode: set to 1, which means the default manual review mode; 6. auto_mode: Set to an empty list, no automatic review is involved.

[0039] Therefore, the step text of test step 1 can be analyzed through the first model to obtain the above parameters and then configure the above request body, so as to call the execution interface based on the request body to create creative content and obtain the processing results of the execution interface, such as obtaining the advertising ID and creative content ID1 and creative content ID2.

[0040] Afterwards, the execution result can be obtained by comparing the processing result with the step text. If the advertisement ID, creative content ID1, and creative content ID2 are obtained, it means that the test step has been tested successfully. If the advertisement ID, creative content ID1, and creative content ID2 are not obtained, it means that the test step has failed.

[0041] As another example, for step 4 in the above test case, a request body can be constructed using the execution agent and execution parameters corresponding to step 4. Where the execution agent determined by the first model is a log query agent and the execution parameter is a creative identifier, an SQL statement can be constructed based on the creative identifier (e.g., ID1, ID2) and the status identifier, thereby invoking the log query interface to implement the log query and obtain the data returned by the query as the processing result. In this embodiment, the log query agent can determine its execution result based on the number of data records contained in the returned data. If the processing result contains n records, the test step is successful; if the processing result does not contain n records, the test step fails.

[0042] In some possible embodiments, the test case includes multiple test steps, and step 12 and step 13 are executed in sequence according to the order of the test steps until each test step is completed to obtain the execution result of the test case.

[0043] Thus, through the above technical solution, users can write test cases in natural language, which allows the model to identify the test agent used to execute the test steps. The test agent then calls the corresponding execution interface to implement the test process of the test steps. This process eliminates the need for extensive human effort in understanding and coding. The model analyzes the description text to call the test agent, allowing batch execution of natural language test cases, effectively reducing the manual workload of testers. Furthermore, testers only need to write natural language test cases, avoiding the need for different users to write test code in different styles. This reduces the complexity of subsequent maintenance and test attribution analysis, lowers the technical requirements for testers, and improves testing efficiency.

[0044] In some possible embodiments, the test case includes multiple test steps, and determining, from multiple test agents, execution agents and execution parameters corresponding to the test steps based on the description text and the first model may include: If the test step is the first step in the test case, the execution agent and the execution parameters are determined based on the step text corresponding to the description text of the test step and the first model.

[0045] The test steps can be executed sequentially according to the order in which they appear in the test case, with the next test step executed after each test step. For the first test step, the step text and description of the test agent can be input into the first model, so that the first model can determine the execution agent to execute the test step and the execution parameters required for the execution agent to call the execution interface.

[0046] If the test step is not the first step in the test case, obtaining the execution results of the executed test steps in the test case; The execution agent and the execution parameters are determined based on the step text, the execution results of the executed test steps and the first model.

[0047] If the test step is not the first step, it means that there is a test step that has been executed before this test step. In this case, the execution result of the executed test step can be used as the context information of the current test step, and then it can be input into the first model together with the step text and the description information of the test agent, so that the first model can analyze and process it to determine the execution agent and execution parameters.

[0048] Therefore, through the above technical solution, when determining the execution agent corresponding to other steps of the test case except the first step, the execution results of the previously executed test steps can be referred to for selection, providing more comprehensive information for determining the execution agent of the current test step, thereby improving the accuracy of the execution agent.

[0049] In some possible embodiments, determining the execution agent and execution parameters corresponding to the test step from a plurality of test agents based on the description text and the first model may include: Determine a classification identifier corresponding to the test case, wherein the classification identifier may be an identifier specified for the test case when the test case is submitted.

[0050] Due to the continuous access of test agents, the first model will have a large number of choices when determining the execution agent based on the step text of the test step, which may affect stability. To ensure the accuracy of the execution agent determination, the classification identifier corresponding to the test agent can be pre-configured to distinguish test agents under different categories. The classification identifier can indicate the business line classification. For example, when writing a test case for the advertising business, the classification identifier corresponding to the test case can be specified as the advertising business identifier.

[0051] A candidate agent is determined from the plurality of test agents according to the classification identifier.

[0052] For example, for data construction agent A1, its configured classification identifier is L1, for data construction agent A2, its configured classification identifier is L2, for database query agent A3, its configured classification identifier is L1, for database query agent A4, its configured classification identifier is L2, for audit log query agent A5, its configured classification identifier is L0, where L0 is a general identifier, indicating that A5 can be used to execute test cases under L1 and L2.

[0053] Accordingly, determining a candidate agent from the plurality of test agents according to the classification identifier may include: The test agent associated with the classification identifier and the test agent associated with the general identifier are determined as the candidate agents.

[0054] The test agent associated with the universal identifier is used to execute the test steps under each classification identifier.

[0055] When configuring a test agent, you can also configure the category identifiers supported by the test agent. For example, if the category identifier of the test agent is not configured, it can be set to a universal identifier by default, that is, it can be used for test cases under various categories.

[0056] As an example, if the classification identifier corresponding to the determined test case is L1, then the test agents A1 and A3 associated with the classification identifier L1 and the test agent A5 associated with the general identifier L0 may be determined as the candidate agents.

[0057] Therefore, through the above technical solution, the first model can make a preliminary selection based on the classification identification of the test agent when determining the execution agent to determine a set of candidate agents, so that the first model can more accurately and easily determine the execution agent used to execute the current test step, improve the consistency between the determined execution agent and the user's test intention, thereby achieving the stability of the user's test intention identification in multiple classification scenarios, improving the accuracy of the execution agent and thus improving the test efficiency of the test step.

[0058] After the candidate agents are determined, the execution agent and the execution parameters may be determined according to the description text, the first model, and the candidate agents.

[0059] For example, the step text of the current test step and the description information of the candidate agents can be used to construct the input prompt text of the first model, so that the first model can determine the agent text that executes the test step from the candidate agents. This can effectively reduce the data processing volume of the first model, improve the efficiency of the first model in determining the execution agent and execution, and also improve the accuracy of the execution agent, thereby ensuring the effective execution of the test steps.

[0060] As another example, determining the execution agent and the execution parameters based on the description text, the first model and the candidate agents may include: if the test step is the first step in the test case, then determining the execution agent and the execution parameters based on the step text corresponding to the test step in the description text, the first model and the candidate agents; if the test step is not the first step in the test case, obtaining the execution result of the executed test step in the test case; and determining the execution agent and the execution parameters based on the step text, the execution result of the executed test step, the first model and the candidate agents. The implementation of the above steps has been described in detail above and will not be repeated here. In this way, the number of candidate agents in each test step can be reduced, the efficiency of determining the execution agent can be improved, and the execution result of the previous step can be combined as context information to improve the accuracy of the execution agent and the execution parameters.

[0061] In some possible embodiments, the method may further include: Based on the execution parameters, the execution result, the step text corresponding to the description text of the test step and the second model, the matching result of the execution result and the step text is determined, wherein the matching result includes a matching success mark, or the matching result includes a matching failure mark and test suggestion information.

[0062] When the execution agent tests a test step, due to the capabilities of the agent model, the execution result may not match the test step, that is, the execution of the test step may not meet expectations. If the execution result of the current test step does not meet expectations, it may affect the execution results of subsequent steps. In this embodiment, after obtaining the execution result of the test step, it can be further determined whether the execution result matches the test step.

[0063] As an example, the second model may be implemented based on an LLM model. Accordingly, determining a matching result between the execution result and the step text based on the execution parameters, the execution result, the step text corresponding to the description text of the test step, and the second model may include: An input prompt text corresponding to the execution agent and a request body for the execution agent to call the execution interface based on the execution parameters are obtained.

[0064] Among them, when the execution agent calls the execution interface based on the execution parameters, it will construct the request body based on the request body structure of the execution interface and the execution parameters, such as adding the execution parameters to the field corresponding to the execution parameters in the request body structure to obtain the request body, and then the execution agent implements the call of the execution interface based on the request body.

[0065] Afterwards, the input prompt text, the execution parameters, the request body, the step text and the execution result are input into the second model to obtain the matching result.

[0066] Among them, the input prompt text prompt of the execution agent will constrain the logic of the execution agent in constructing the request body, and the input prompt text, the execution parameters, the request body, the step text and the execution result are input into the second model, so that the second model can determine whether the request body constructed by the execution agent is valid based on the input prompt text and the execution parameters, and determine whether the execution result meets the execution expectations of the test step based on the step text and the execution result.

[0067] As an example, if the second model determines that the execution result and the step text match successfully, the matching result output by the second model includes a matching success flag; if the second model determines that the execution result and the step text match fails, the matching result output by the second model includes a matching failure flag and test suggestion information, wherein the test suggestion information is used to indicate the update parameters of the execution interface called by the execution agent based on the execution parameters.

[0068] Taking step 1 in the above test case as an example, after obtaining the execution result of step 1, the input prompt text, execution parameters, request body, step text, and execution result of step 1 are input into the second model to determine that the execution result and the step text fail to match. The step text is constrained to create a type C creative content, while the execution result creates a type B creative content. Based on the input prompt text, execution parameters, and request body, a matching analysis is performed to determine that b_line in the constructed request body is 0. Based on the above, b_line 1 indicates creative content of type C. The test modification information can be b_line:1. The matching result then includes a match failure indicator and the test modification information b_line:1.

[0069] If the matching result indicates that the execution result and the step text fail to match, a new execution result of the test step is determined based on the test suggestion information in the matching result, the execution parameters, and the execution agent.

[0070] In this step, the test suggestion information, execution parameters, and input prompt text can be constructed into the input of the execution agent so that the execution agent can construct the correct request body. For example, in the request body constructed this time, b_line is 1, thereby creating creative content under type C and achieving accurate execution of the test steps.

[0071] If the matching result indicates that the execution result and the step text match successfully, and the next test step of the test step exists in the test case, the next test step will be used as a new test step, and the execution of the step and its subsequent steps of determining the execution agent and execution parameters corresponding to the test step from multiple test agents based on the description text and the first model is returned to, so that the next test step can be executed after determining that the current test step is successfully executed, avoiding the impact of the failure of the current test step on the subsequent steps, and avoiding invalid test process execution.

[0072] If the matching result indicates that the execution result and the step text match successfully, and the next test step of the test step does not exist in the test case, the test case execution result of the test case is obtained.

[0073] Therefore, through the above technical solution, the execution results of the test agent can be monitored based on the model to determine whether the execution results are valid, and if the execution results are incorrect, the test suggestion information can be given to the execution agent to allow the execution agent to re-execute, thereby improving the stability of the execution agent and at the same time improving the accurate execution of the test steps in the test case, ensuring the accurate testing of the test case, and improving the stability and effectiveness of the test process.

[0074] In some possible embodiments, the method may further include: After obtaining the use case execution result of the test case, code is generated based on the execution process of the test steps in the test case to obtain the execution code corresponding to the test case, wherein the use case execution result includes the execution results of the test steps in the test case.

[0075] After a test step is executed, the execution logic of the test step and the request body and interface return data of the corresponding execution interface can be recorded. After the test case is executed, the execution logic corresponding to the test step in the test case and the request body and interface return data of the execution interface corresponding to the test step can be obtained. After that, a code generation agent can be called to generate execution code. For example, based on the execution logic of the test step in the test case, the request body and interface return data of the execution interface, an input prompt can be constructed through the prompt text prompt of the code generation agent to input the code generation agent, thereby obtaining the execution code corresponding to the test case.

[0076] Therefore, the execution code corresponding to the test case can be automatically generated after the test case is executed, and the test case can be solidified so that the subsequent regression test can be directly based on the execution code to improve the test efficiency.

[0077] In some possible embodiments, the method further includes: After obtaining the test case execution results, a test report is generated based on the execution results of the test steps in the test case. The test report can be generated based on a report generation agent. Thus, a user can clearly understand the execution process of each test step based on the test report, so as to adjust the test case or locate anomalies.

[0078] like Figure 2 As shown, it is a flow chart of a testing method provided according to an embodiment of the present disclosure, as shown in Figure 2 As shown, a tester writes a test case in natural language to obtain a description text and submits a test task based on this description text. The task management platform then receives the test task and extracts the test steps based on the test task description text, obtaining the step text and classification identifiers of the test steps. It also obtains the classification identifiers and description information of the test agents A1-An. The classification identifiers, execution interfaces, and description information of the test agents can be pre-registered. They can then be grouped based on different classification identifiers to obtain agents with classification identifiers 1-m. The first model then performs intent recognition based on the step text, the test case classification identifier, and the test agent classification identifiers and description information to determine the execution agent and execution parameters for executing the test step. The engine then sends the step text to the execution agent, which returns an execution result. The second model further determines whether the execution result matches the step text. If not, a match failure indicator and test suggestion information are returned to the engine. The engine then sends the step text and test suggestion information to the execution agent for re-execution to obtain an execution result. The second model further determines whether the execution result matches the step text. If so, the test step is considered complete and the next test step is executed. The specific implementation methods of the above steps have been described in detail above and will not be repeated here.

[0079] In some possible embodiments, the method further includes: The test case is displayed in a display interface.

[0080] As an example, the test cases may be displayed in a list format in the display interface, such as the test case title, status, and test case execution result, to provide an intuitive display for the tester, making it easier for the tester to view the test case.

[0081] In response to a selection operation on the test case, test information corresponding to the test step in the test case is displayed, wherein the test information includes the step text of the test step in the description text and the execution result of the test step.

[0082] If the user wants to view the specific content of one of the test cases, he can select the test case, such as by clicking to perform the selection operation. The selection operation method can be configured based on the actual application scenario, and this disclosure does not limit this. Figure 3 As shown, when a user selects a test case, such as the test case titled XX1, the step text (as shown in column L1) and the execution results (as shown in column L2) of the test steps in the test case can be displayed in the area in the display interface.

[0083] As another example, to provide more information to the user, the return data of the execution agent corresponding to the test step can also be displayed, as shown in column L3. For another example, the status of the test step can also be displayed, as shown in column L4, to indicate to the user whether the test step has been completed.

[0084] Therefore, through the above technical solution, a visual display can be made for users, and test cases can be displayed at the granularity of test steps, so that testers can determine the anomaly more quickly and accurately when the test fails, providing effective data support for testers to troubleshoot test anomalies.

[0085] In some possible embodiments, the test step is also associated with displaying an edit control, such as Figure 3 As shown in B1 in . Accordingly, the method may further include: In response to a selection operation on the edit control of the test step, a step text of the test step is displayed.

[0086] If the user finds that the test steps are incorrectly written after the test case is completed or during the test, the user can re-edit the step text through the editing control. For example, the user can click the B1 control in step 5 to display the step text of the test step. If the user edits step 5, the step text of step 5 "Send queue message, the message format is {XXX1}" can be displayed.

[0087] In response to an editing operation on the step text, the edited text is used as a new step text of the test step.

[0088] Afterwards, the user can perform editing operations in the displayed interface. For example, if the edited text is "Send queue message, message format is {XXX2}", the step text of step 5 will be "Send queue message, message format is {XXX2}".

[0089] An execution result of the test step is determined based on the new step text of the test step, the first model, and the test agent.

[0090] Since the step text of step 5 has been updated, but the step text of the previous test step has not been updated, the test step 5 can be re-executed, such as determining the execution agent and execution parameters corresponding to the test step from multiple test agents based on the new step text of the test step and the first model, and then determining the new execution result of the test step based on the execution agent, the execution parameters and the execution interface.

[0091] If the test step is not the last test step of the test case, after obtaining the execution result of the test step, the execution result of the test step after the test step in the test case is re-determined.

[0092] If the test step is not the last test step of the test case, its execution result may affect the execution process of subsequent test steps. In this embodiment, the execution result of the test step after the test step in the test case is re-determined to achieve the re-execution of subsequent steps.

[0093] Therefore, through the above technical solution, testers can edit a certain test step in a test case and re-execute the test case from that test step, ensuring the stable execution of the test case. At the same time, it reduces the resources and time required to re-execute the test case after modification to a certain extent, thereby improving testing efficiency.

[0094] In some possible embodiments, the test step is also associated with displaying an insert control, such as Figure 3 As shown in B2 and B3 in FIG, B2 is used to indicate upward insertion, i.e., inserting a step before the current step, and B3 indicates downward insertion, i.e., inserting a step after the current step. Accordingly, the method may further include: In response to a selection operation on the insert control of the test step, an editing window of a new test step is displayed, wherein the editing window may be displayed in a floating layer manner.

[0095] If the user finds that one or more steps need to be added after the test case is completed or during the test, the user can add test steps by inserting controls. For example, the user can click the B2 control in step 5 to display an editing window, in which the user can edit the step text of the test step.

[0096] The order of the new test steps in the test case is determined based on the type of the inserted control, and the execution result of the new test step is determined based on the step text of the new test step, the first model, and the test agent. If the B2 control is inserted upward, this indicates that the new test step is inserted before step 5. That is, the new test step is inserted as step 5, and the original order of steps 5-8 in the test case is postponed to 6-9.

[0097] If the new test step is not the last test step in the test case, then after obtaining the execution result of the new test step, the execution result of the test step following the new test step in the test case is re-determined. The method for determining the execution result of the test step has been described in detail above and will not be repeated here.

[0098] Therefore, through the above technical solution, testers can insert new test steps into test cases, edit test cases, and re-execute test cases starting from the new test steps, which to a certain extent reduces the resources and time required to re-execute test cases after modification and improves testing efficiency.

[0099] Based on the same inventive concept, the present disclosure also provides a testing device, such as Figure 4 As shown, the device 10 includes: The receiving module 100 is configured to receive a description text of a test case, wherein the description text is a text written based on a natural language, and the test case includes at least one test step; A first determining module 200 is configured to determine, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents, wherein the test agent is configured to call an execution interface corresponding to the test agent; The first testing module 300 is configured to determine an execution result of the test step based on the execution agent, the execution parameters, and the execution interface.

[0100] Optionally, the device further comprises: a second determination module, configured to determine a matching result between the execution result and the step text based on the execution parameter, the execution result, a step text corresponding to the description text of the test step, and a second model, wherein the matching result includes a matching success flag, or the matching result includes a matching failure flag and test suggestion information; The second test module is used to determine the new execution result of the test step based on the test suggestion information, the execution parameters and the execution agent in the matching result if the matching result indicates that the execution result and the step text match fails; if the matching result indicates that the execution result and the step text match successfully, and the next test step of the test step exists in the test case, then the next test step is used as the new test step, triggering the first determination model to determine the execution agent and execution parameters corresponding to the test step from multiple test agents based on the description text and the first model.

[0101] Optionally, the second determining module includes: An acquisition submodule, configured to acquire an input prompt text corresponding to the execution agent and a request body for the execution agent to call the execution interface based on the execution parameters; The processing submodule is used to input the input prompt text, the execution parameter, the request body, the step text and the execution result into the second model to obtain the matching result.

[0102] Optionally, the test case includes multiple test steps, and the first determining module includes: a first determining submodule, configured to determine, if the test step is the first step in the test case, the execution agent and the execution parameters based on the step text corresponding to the test step in the description text and the first model; A second determining submodule is configured to obtain an execution result of an executed test step in the test case if the test step is not the first step in the test case; The third determination submodule is configured to determine the execution agent and the execution parameters based on the step text, the execution results of the executed test steps, and the first model.

[0103] Optionally, the first determining module includes: A fourth determination submodule is used to determine a classification identifier corresponding to the test case; a fifth determining submodule, configured to determine a candidate agent from the plurality of test agents according to the classification identifier; A sixth determination submodule is configured to determine the execution agent and the execution parameters based on the description text, the first model, and the candidate agents.

[0104] Optionally, the fifth determining submodule is further configured to: determining the test agent associated with the classification identifier and the test agent associated with the general identifier as the candidate agents; The test agent associated with the universal identifier is used to execute the test steps under each classification identifier.

[0105] Optionally, the device further comprises: A first display module is used to display the test case in a display interface; The third test module is used to display the test information corresponding to the test step in the test case in response to the selection operation of the test case, wherein the test information includes the step text of the test step in the description text and the execution result of the test step.

[0106] Optionally, the test step is further associated with displaying an edit control; The device further comprises: A second display module, configured to display the step text of the test step in response to a selection operation on the edit control of the test step; a third determining module, configured to, in response to an editing operation on the step text, use the edited text as a new step text of the test step; a fourth testing module, configured to determine an execution result of the test step based on the new step text of the test step, the first model, and the test agent; The fifth testing module is used to, if the test step is not the last test step of the test case, redetermine the execution result of the test step after the test step in the test case after obtaining the execution result of the test step.

[0107] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device (e.g., a terminal device or a server) 600 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0108] like Figure 5As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or programs loaded from a storage device 608 into a random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0109] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Figure 5 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0110] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0111] It should be noted that the computer-readable medium described above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wire, optical cable, RF (radio frequency), or any suitable combination thereof.

[0112] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0113] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0114] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: receives a description text of a test case, wherein the description text is a text written based on a natural language, and the test case contains at least one test step; based on the description text and the first model, determines the execution agent and execution parameters corresponding to the test step from multiple test agents, wherein the test agent is used to call the execution interface corresponding to the test agent; based on the execution agent, the execution parameters and the execution interface, determines the execution result of the test step.

[0115] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a module does not necessarily define the module itself. For example, a receiving module may also be described as a "module that receives a description text of a test case."

[0118] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0119] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] According to one or more embodiments of the present disclosure, Example 1 provides a testing method, the method comprising: receiving a description text of a test case, wherein the description text is a text written based on a natural language, and the test case includes at least one test step; Determining, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents, wherein the test agent is used to call an execution interface corresponding to the test agent; An execution result of the test step is determined based on the execution agent, the execution parameters, and the execution interface.

[0121] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, further comprising: Based on the execution parameters, the execution result, the step text corresponding to the description text of the test step, and the second model, determining a matching result between the execution result and the step text, wherein the matching result includes a matching success flag, or the matching result includes a matching failure flag and test suggestion information; If the matching result indicates that the execution result and the step text fail to match, determining a new execution result of the test step based on the test suggestion information, the execution parameters, and the execution agent in the matching result; If the matching result indicates that the execution result and the step text match successfully, and the next test step of the test step exists in the test case, the next test step is used as a new test step, and the execution is returned to the step of determining the execution agent and execution parameters corresponding to the test step from multiple test agents based on the description text and the first model.

[0122] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein determining a matching result between the execution result and the step text based on the execution parameter, the execution result, the step text corresponding to the test step in the description text, and the second model includes: Obtaining an input prompt text corresponding to the execution agent and a request body for the execution agent to call the execution interface based on the execution parameters; The input prompt text, the execution parameter, the request body, the step text and the execution result are input into the second model to obtain the matching result.

[0123] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 1, wherein the test case includes multiple test steps, and determining, based on the description text and the first model, execution agents and execution parameters corresponding to the test steps from multiple test agents includes: If the test step is the first step in the test case, determining the execution agent and the execution parameters based on the step text corresponding to the test step in the description text and the first model; If the test step is not the first step in the test case, obtaining the execution results of the executed test steps in the test case; The execution agent and the execution parameters are determined based on the step text, the execution results of the executed test steps and the first model.

[0124] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 1, wherein determining, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents includes: Determine the classification identifier corresponding to the test case; determining a candidate agent from the plurality of test agents according to the classification identifier; The execution agent and the execution parameters are determined according to the description text, the first model and the candidate agent.

[0125] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, wherein determining a candidate agent from the plurality of test agents according to the classification identifier includes: determining the test agent associated with the classification identifier and the test agent associated with the general identifier as the candidate agents; The test agent associated with the universal identifier is used to execute the test steps under each classification identifier.

[0126] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 1, further comprising: Displaying the test case in a display interface; In response to a selection operation on the test case, test information corresponding to the test step in the test case is displayed, wherein the test information includes the step text of the test step in the description text and the execution result of the test step.

[0127] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 7, wherein the testing step is further associated with displaying an edit control; The method further comprises: In response to a selection operation on an edit control of the test step, displaying a step text of the test step; In response to an editing operation on the step text, using the edited text as a new step text for the test step; determining an execution result of the test step based on the new step text of the test step, the first model, and the test agent; If the test step is not the last test step of the test case, after obtaining the execution result of the test step, the execution result of the test step after the test step in the test case is re-determined.

[0128] According to one or more embodiments of the present disclosure, Example 9 provides a testing device, the device comprising: A receiving module, configured to receive a description text of a test case, wherein the description text is a text written based on a natural language, and the test case includes at least one test step; a first determining module configured to determine, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents, wherein the test agent is configured to call an execution interface corresponding to the test agent; The first testing module is configured to determine an execution result of the test step based on the execution agent, the execution parameters, and the execution interface.

[0129] According to one or more embodiments of the present disclosure, Example 10 provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in any one of Examples 1-8 when executed by a processing device.

[0130] According to one or more embodiments of the present disclosure, Example 11 provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method described in any one of Examples 1-8.

[0131] According to one or more embodiments of the present disclosure, Example 12 provides a computer program product, including a computer program, which implements the steps of any one of the methods of Examples 1-8 when executed by a processor.

[0132] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the scope of the above disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0133] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0134] Although the subject matter has been described using language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.

Claims

1. A testing method, characterized in that: The method comprises: receiving a description text of a test case, wherein the description text is a text written based on a natural language, and the test case includes at least one test step; Determining, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents, wherein the test agent is used to call an execution interface corresponding to the test agent; An execution result of the test step is determined based on the execution agent, the execution parameters, and the execution interface.

2. The method according to claim 1, characterized in that The method further comprises: Based on the execution parameters, the execution result, the step text corresponding to the description text of the test step, and the second model, determining a matching result between the execution result and the step text, wherein the matching result includes a matching success flag, or the matching result includes a matching failure flag and test suggestion information; If the matching result indicates that the execution result and the step text fail to match, determining a new execution result of the test step based on the test suggestion information, the execution parameters, and the execution agent in the matching result; If the matching result indicates that the execution result and the step text match successfully, and the next test step of the test step exists in the test case, the next test step is used as a new test step, and the execution is returned to the step of determining the execution agent and execution parameters corresponding to the test step from multiple test agents based on the description text and the first model.

3. The method according to claim 2, characterized in that The determining a matching result between the execution result and the step text based on the execution parameter, the execution result, the step text corresponding to the description text of the test step, and the second model includes: Obtaining an input prompt text corresponding to the execution agent and a request body for the execution agent to call the execution interface based on the execution parameters; The input prompt text, the execution parameter, the request body, the step text and the execution result are input into the second model to obtain the matching result.

4. The method according to claim 1, wherein The test case includes multiple test steps, and determining execution agents and execution parameters corresponding to the test steps from multiple test agents based on the description text and the first model includes: If the test step is the first step in the test case, determining the execution agent and the execution parameters based on the step text corresponding to the test step in the description text and the first model; If the test step is not the first step in the test case, obtaining the execution results of the executed test steps in the test case; The execution agent and the execution parameters are determined based on the step text, the execution results of the executed test steps and the first model.

5. The method according to claim 1, wherein The determining, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents includes: Determine the classification identifier corresponding to the test case; determining a candidate agent from the plurality of test agents according to the classification identifier; The execution agent and the execution parameters are determined according to the description text, the first model and the candidate agent.

6. The method according to claim 5, characterized in that Determining a candidate agent from the plurality of test agents according to the classification identifier includes: determining the test agent associated with the classification identifier and the test agent associated with the general identifier as the candidate agents; The test agent associated with the universal identifier is used to execute the test steps under each classification identifier.

7. The method according to claim 1, characterized in that The method further comprises: Displaying the test case in a display interface; In response to a selection operation on the test case, test information corresponding to the test step in the test case is displayed, wherein the test information includes the step text of the test step in the description text and the execution result of the test step.

8. The method according to claim 7, characterized in that The test step is also associated with displaying an edit control; The method further comprises: In response to a selection operation on an edit control of the test step, displaying a step text of the test step; In response to an editing operation on the step text, using the edited text as a new step text for the test step; determining an execution result of the test step based on the new step text of the test step, the first model, and the test agent; If the test step is not the last test step of the test case, after obtaining the execution result of the test step, the execution result of the test step after the test step in the test case is re-determined.

9. A testing device, characterized in that: The device comprises: A receiving module, configured to receive a description text of a test case, wherein the description text is a text written based on a natural language, and the test case includes at least one test step; a first determining module configured to determine, based on the description text and the first model, an execution agent and execution parameters corresponding to the test step from a plurality of test agents, wherein the test agent is configured to call an execution interface corresponding to the test agent; The first testing module is configured to determine an execution result of the test step based on the execution agent, the execution parameters, and the execution interface.

10. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images