Test methods, equipment, storage media and program products

Abstract

The embodiments of the present application provide a test method, device, storage medium and program product, which are applied to the field of software testing. It includes determining multiple test cases of an object to be tested and the dependencies between the multiple test cases; constructing a directed acyclic graph with the multiple test cases as nodes and the dependencies between the multiple test cases as edges; executing the test cases corresponding to the multiple nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested; determining the abnormal type of the abnormal event in response to an abnormal event; and executing the corresponding fault handling operation according to the recovery method corresponding to the abnormal type. The scheme of the embodiment of the present application improves the rationality of the execution of test cases and improves the test efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of testing, and in particular to a testing method, device, storage medium, and program product. Background Art

[0002] In order to facilitate the testing of objects such as system software and application software and to improve the testing efficiency, automated testing systems are developed.

[0003] In this type of testing system, testers only need to write test cases (a set of input data, execution conditions and expected results, a test plan and steps used to verify whether the software meets the requirements) to automatically implement software testing to discover errors and defects in the software development process, detect whether the software has completed the intended functions, and met the expected requirements, etc.

[0004] In traditional solutions, multiple test cases are usually written for a test object, and the test system executes the multiple test cases one by one, and then determines whether the test object meets specific requirements based on the execution results of the multiple test cases. However, this testing method takes a long time when there are a large number of test cases, has low testing efficiency, and can only be intervened manually in the event of a failure. Therefore, a reasonable testing method is urgently needed. Summary of the invention

[0005] The embodiments of the present application provide a testing method, device, storage medium and program product to solve the problem of unreasonable software testing methods in traditional solutions.

[0006] In a first aspect, an embodiment of the present application provides a testing method, comprising:

[0007] Determine multiple test cases for the object to be tested and dependencies between the multiple test cases;

[0008] Constructing a directed acyclic graph by using the multiple test cases as nodes and the dependency relationships between the multiple test cases as edges;

[0009] Execute the test cases corresponding to the multiple nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested;

[0010] In response to an abnormal event, determining an abnormal type of the abnormal event;

[0011] Perform corresponding fault handling operations according to the recovery method corresponding to the exception type.

[0012] In a second aspect, an embodiment of the present application provides a testing method, comprising:

[0013] Determine multiple test cases for the object to be tested and dependencies between the multiple test cases;

[0014] Constructing a directed acyclic graph by using the multiple test cases as nodes and the dependency relationships between the multiple test cases as edges;

[0015] According to the node execution order indicated by the directed acyclic graph, the test cases corresponding to the multiple nodes are executed to test the object to be tested.

[0016] In a third aspect, an embodiment of the present application provides a computing device, comprising a storage component and a processing component; the storage component stores one or more computer program instructions, the computer program instructions are called and executed by the processing component, and the processing component executes the one or more computer program instructions to implement the test method described in the first aspect, or the test method described in the second aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a computer to implement the test method described in the first aspect or the test method described in the second aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product storing a computer program, wherein when the computer program is executed by a computer, the test method described in the first aspect or the test method described in the second aspect is implemented.

[0019] In the embodiment of the present application, a directed acyclic graph is constructed by using multiple test cases as nodes and the dependencies between the test cases as edges, and the test cases corresponding to the multiple nodes are executed in the node execution order indicated by the directed acyclic graph to test the test object. By constructing a directed acyclic graph, efficient and orderly test case execution is achieved, and the rationality of the test is improved. On this basis, it is also possible to respond to abnormal events, determine the abnormal type of the abnormal event, and perform corresponding fault handling operations according to the recovery method corresponding to the abnormal type, without manual intervention, that is, to achieve fault handling, ensure the efficiency and accuracy of fault handling, further improve test efficiency, and improve test rationality.

[0020] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flow chart of an embodiment of a testing method provided by the present application is shown;

[0023] Figure 2 A structural schematic diagram of a directed acyclic graph in practical applications is shown;

[0024] Figure 3 A flow chart showing another embodiment of a testing method provided by the present application is shown;

[0025] Figure 4 A schematic structural diagram of an embodiment of a testing device provided by the present application is shown;

[0026] Figure 5 A structural schematic diagram of another embodiment of a testing device provided by the present application is shown;

[0027] Figure 6 A system architecture diagram of a test system in practical application is shown;

[0028] Figure 7 A schematic diagram of the structure of an embodiment of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0030] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0031] The technical solution of the embodiment of the present application can be applied to the field of software testing, and is particularly applicable to the fault handling scenario in the process of software testing. As described in the background technology, the current testing method of executing multiple test cases one by one often waits until the previous test case is completed before executing the next test case. When the number of test cases is large, it takes a long time and the test efficiency is low.

[0032] In order to improve the test efficiency, the inventors have come up with the idea of ​​executing multiple test cases simultaneously. However, this test method may cause deadlock due to uneven scheduling of test resources. For example, if two test cases are executed simultaneously, and one test case requires the execution result of another test case, then if the required test case has not been completed, the test case can only wait and cannot continue to execute. When multiple test cases fall into a loop of waiting, deadlock may occur.

[0033] In addition, the execution of test cases may be affected by various factors such as network connection failure, system operation failure, etc., resulting in abnormalities and interruption of test execution. According to the above test method, manual fault recovery is usually adopted, and the test user manually searches for the test case to be recovered, which has low accuracy and low efficiency.

[0034] In order to solve the above technical problems, the inventors proposed the technical solution of the present application, including: determining multiple test cases for an object to be tested and the dependencies between the multiple test cases; constructing a directed acyclic graph with the multiple test cases as nodes and the dependencies between the multiple test cases as edges; executing the test cases corresponding to the multiple nodes respectively in the order of node execution indicated by the directed acyclic graph to test the object to be tested; determining the exception type of the exception event in response to an abnormal event; and performing corresponding fault handling operations in accordance with the recovery method corresponding to the exception type.

[0035] The scheme of the embodiment of the present application is to construct a directed acyclic graph by using multiple test cases as multiple nodes and the dependency relationship between the test cases as the edge, and execute the test cases corresponding to the multiple nodes according to the node execution order indicated by the directed acyclic graph to test the object to be tested. By constructing a directed acyclic graph, the problem of the traditional scheme of executing multiple test cases one by one and then judging whether the object to be tested meets specific requirements according to the execution results of the multiple test cases is solved, which leads to the problem of long time consumption and low test efficiency when the number of test cases is large, and it can avoid the deadlock and other problems that may occur when executing multiple test cases at the same time, realize efficient and orderly test case execution, and improve the rationality of the test. On this basis, it is also possible to respond to abnormal events, determine the abnormal type of the abnormal event, and execute the corresponding fault handling operation according to the recovery method corresponding to the abnormal type, without manual intervention, that is, to realize fault handling, ensure the efficiency and accuracy of fault handling, further improve the test efficiency, and improve the rationality of the test.

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0037] The technical solution of the embodiment of the present application can be applied to a system architecture including a test end. The test end can provide a configuration interface to support testers in configuring test cases, and can also be provided with a test machine to execute test cases, etc.

[0038] Optionally, the system may further include a configuration terminal, and the configuration terminal may establish a connection with the test terminal through a network. The network is a medium that provides a communication link between the configuration terminal and the test terminal. The network may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The configuration terminal may interact with the test terminal through the network to send configuration information, etc.

[0039] The configuration end can provide a configuration interface to support testers to configure test cases. The configuration end can be a browser, APP (Application), or web application such as H5 (Hyper Text Markup Language5, Hypertext Markup Language Version 5) application, or light application (also known as applet, a lightweight application) or cloud application, etc. The user end can be deployed in an electronic device and needs to rely on the device to run or some apps in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, tablet computer, personal computer, etc. Various other types of applications can usually be configured in electronic devices, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0040] The test end may include a server that provides various services, such as a server that processes configuration information sent by the configuration end, a server that executes tests, etc.

[0041] It should be noted that the test end can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0042] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described in this article within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0043] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0044] It should be noted that the technical solution of the embodiment of the present application is applicable to a network virtual environment. The users described generally refer to "virtual users". Real users can register a user account in the server through registration to obtain a user identity in the network environment.

[0045] like Figure 1As shown, it is a flow chart of an embodiment of a testing method provided by the present application, which may include the following steps.

[0046] 101: Determine multiple test cases for the object to be tested and the dependencies between the multiple test cases.

[0047] Among them, the object to be tested may refer to the software to be tested, and the test case may be pre-configured and generated by the tester based on the test requirements of the object to be tested. For each test case, its configured test attributes may also be determined, such as test content, test function, required test resources, test machine, execution time, and execution priority. The required test resources may include hardware resources such as the CPU (Central Processing Unit) and memory required to execute the test case, and may also include software resources such as an operating system. The test machine may refer to a virtual machine that executes the test case, etc. The execution time may refer to the estimated time to execute the test case. The execution priority may be used to assist in the subsequent construction of a directed acyclic graph, such as the test case with the highest execution priority may correspond to the start node in the directed acyclic graph, etc.

[0048] In the embodiment of the present application, the dependency relationship between multiple test cases to be tested can also be determined. For two test cases with a dependency relationship, the execution result of the dependent test case will be used as the input data of the other test case. It is understandable that a test case can depend on multiple test cases, that is, the input data of the test case requires the execution results of multiple test cases. In addition, a test case can also be relied on by multiple test cases, that is to say, the execution result of the test case can be used as the input data of multiple test cases. Among them, the dependency relationship can also be pre-configured and generated by the tester based on the test requirements and the test attributes of each test case.

[0049] 102: Construct a directed acyclic graph with multiple test cases as nodes and dependencies between multiple test cases as edges.

[0050] Directed Acyclic Graph (DAG) is a data structure, a directed graph without loops, consisting of nodes and edges, each edge pointing from one node to another. DAG has connectivity, that is, starting from a node, you can reach another node through a series of edges. At the same time, there is no loop in DAG, that is, starting from any node, you cannot return to the starting node through a series of edges. It is widely used in topological sorting, path analysis and other fields.

[0051] In an embodiment of the present application, when executing a test task for an object to be tested, multiple test cases of the object to be tested can be used as nodes, and the dependency relationship between multiple test cases can be used as edges to construct a DAG structure. Specifically, using the dependency relationship between multiple test cases as an edge can mean that the nodes corresponding to two test cases with a dependency relationship are connected with an edge, and the direction of the edge is from the node corresponding to the dependent test case to the node corresponding to the other test case.

[0052] Optionally, for the convenience of description, for multiple nodes in a DAG structure, identification information corresponding to the multiple nodes may be generated, and description information of the corresponding nodes may be generated according to the test attributes of the test case. The description information may record the test attributes of the corresponding test case.

[0053] In practical applications, DAG structures can be constructed in various forms such as tree diagrams, adjacency matrices, and adjacency lists. For ease of understanding, Figure 2 A schematic diagram of a DAG structure in a practical application is shown. Figure 2 As shown in the figure, the DAG structure includes four nodes a~d, corresponding to four test cases A~D. There is an edge relationship between nodes a and b, and the direction of the edge is from node a to node b, indicating that test case B depends on test case A. There is also an edge relationship between nodes b and d, and between nodes c and d, and the direction of the edge is from b to d, and from c to d, indicating that test case D depends on test cases B and C. There is no edge relationship between nodes a and c, indicating that test cases A and C do not have a dependency relationship.

[0054] 103: Execute the test cases corresponding to the multiple nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested.

[0055] The node execution order means that for two nodes connected by an edge, the node at the beginning of the connection is executed first, and after the execution of the node is completed and the execution result is obtained, the execution result is used as the input data of the node at the end of the connection, and then the node at the end of the connection is executed. Therefore, the test cases corresponding to multiple nodes are executed according to the node execution order. In other words, for two test cases with a dependency relationship, the dependent test case is executed first, and after the execution of the test case is completed and the execution result is obtained, the execution result is used as the input data of another test case, and then the other test case is executed.

[0056] In this embodiment, a directed acyclic graph is constructed by using multiple test cases as nodes and the dependency relationships between test cases as edges, and the test objects are tested by executing the test cases corresponding to the multiple nodes in the order of node execution indicated by the directed acyclic graph. By constructing a directed acyclic graph, efficient and orderly test case execution is achieved, thereby improving test rationality.

[0057] Optionally, the method of executing test cases corresponding to a plurality of nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested may include:

[0058] According to the node execution order indicated by the directed acyclic graph, the test cases corresponding to the multiple nodes are executed respectively, and the test cases corresponding to at least two nodes without dependency relationship are executed in parallel.

[0059] That is to say, two nodes that have no connection relationship can be executed in parallel, that is, two test cases that do not have a dependency relationship can be executed in parallel to speed up the test efficiency.

[0060] By executing the test cases corresponding to multiple nodes with dependencies in sequence according to the node execution order indicated by the directed acyclic graph, and executing the test cases corresponding to at least two nodes without dependencies in parallel, the problem of the traditional solution of executing multiple test cases one by one and then determining whether the object to be tested meets specific requirements based on the execution results of the multiple test cases, which leads to long time consumption and low test efficiency when the number of test cases is large, can be solved. On the basis of improving the test efficiency, it avoids the deadlock caused by multiple test cases waiting in a loop, thereby improving the rationality of the test.

[0061] The following describes the test case configuration process.

[0062] In an optional embodiment, the test end can obtain the configuration information sent by the configuration end. The configuration end can provide a configuration interface. In response to the configuration operation of the tester in the configuration interface, a configuration request is generated and sent to the test end. In response to the configuration request, the configuration end can obtain configuration information such as multiple test cases of the object to be tested, the dependencies between the multiple test cases, and the test attributes corresponding to the multiple test cases. A directed acyclic graph is constructed with the multiple test cases as nodes and the dependencies between the multiple test cases as edges. Figure 1 The test method shown in the examples performs the test.

[0063] In another optional embodiment, the test end itself can provide a configuration interface, and in response to the configuration operation of the tester, determine the configuration information such as multiple test cases of the object to be tested, the dependencies between the multiple test cases, and the test attributes corresponding to the multiple test cases, and construct a directed acyclic graph. Figure 1 The test method shown in the examples performs the test.

[0064] Among them, the configuration interface may include test case configuration controls such as a candidate test case selection list, a test case upload button, and a test attribute input box. Testers can select the required test case from the candidate test case selection list, click the test case upload button to obtain the test case from the local or specified address, or enter the test attributes of the test case, the dependency relationship with other test cases, etc. in the input box to configure the test case.

[0065] In the actual test execution process, the test process may be abnormal due to factors such as network connection failure and system operation failure, which may cause the execution of the test case to be interrupted, and then the overall test execution to be interrupted. Therefore, fault recovery can also be performed. Figure 3 As shown, it is a flowchart of another embodiment of a testing method provided by the present application, which may include the following steps.

[0066] 301: Determine multiple test cases for the object to be tested and dependencies between the multiple test cases.

[0067] 302: A directed acyclic graph is constructed with multiple test cases as nodes and dependencies between the multiple test cases as edges.

[0068] 303: Execute the test cases corresponding to the multiple nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested.

[0069] In this embodiment, the implementation process of steps 301 to 303 can refer to Figure 1 The implementation process of steps 101 to 103 in the illustrated embodiment will not be described in detail here.

[0070] 304: In response to the abnormal event, determine the abnormal type of the abnormal event.

[0071] 305: Execute corresponding fault handling operations according to the recovery method corresponding to the exception type.

[0072] In the embodiment of the present application, corresponding recovery methods can be pre-configured and generated for different exception types. Therefore, during the test execution process, for the abnormal event that occurs, the abnormal type of the abnormal event can be determined first, and the recovery method corresponding to the abnormal type can be found, and the corresponding fault handling operation can be performed according to the recovery method. For example, for the abnormal event that the network connection fails and the test case execution is interrupted, the relevant personnel can be prompted to repair the network and re-execute the test. For another example, for the abnormal event of the test machine failure, the test machine can be switched to continue the test process, and so on.

[0073] The recovery methods corresponding to the above-mentioned different exception types can be configured by the tester when configuring the test case.

[0074] In this embodiment, a directed acyclic graph is constructed by using multiple test cases as nodes and the dependency relationships between the test cases as edges, and the test cases corresponding to the multiple nodes are executed according to the node execution order indicated by the directed acyclic graph to test the object to be tested. By constructing a directed acyclic graph, the problem of the traditional solution of executing multiple test cases one by one and then judging whether the object to be tested meets specific requirements according to the execution results of the multiple test cases is solved, which leads to a long time consumption and low test results when the number of test cases is large, and the problem of deadlock and other problems that may occur when executing multiple test cases at the same time can be avoided, and efficient and orderly test case execution is achieved, and the rationality of the test is improved. On this basis, it is also possible to respond to abnormal events, determine the abnormal type of the abnormal event, and execute the corresponding fault handling operation according to the recovery method corresponding to the abnormal type, without manual intervention, that is, to achieve fault handling, ensure the efficiency and accuracy of fault handling, further improve the test efficiency, and improve the rationality of the test.

[0075] In practical applications, the operation of the test system during the test execution process can also be monitored. Therefore, in some embodiments, the above method can also include:

[0076] During the execution of multiple test cases, detect the operating data of the test system;

[0077] When the operating data meets the abnormal conditions, a corresponding abnormal event is generated.

[0078] The test system may refer to a system for executing test cases, and may include a test machine, etc. The operation data may include performance indicator data corresponding to the test system, and / or operation log data of the test case. The performance indicator data may include, for example, hardware indicator data such as CPU occupancy, memory usage, disk I / O (indicating the frequency and speed of disk read and write operations), and may also include software indicator data such as system response time and throughput. The operation log data of the test case may include, for example, different operation logs such as obtaining input data, calculating the input data to obtain the execution result, etc.

[0079] Specifically, you can use performance monitoring tools, such as monitoring tools with network monitoring functions, monitoring systems that can monitor system operation status and network information, health check programs, service detection programs, etc., to detect the performance indicator data of the test system. You can use log monitoring systems, such as log file management tools, to detect the operation log data of test cases.

[0080] Different abnormal conditions can be set for different operating data, so that when the corresponding abnormal conditions are met, corresponding abnormal events can be generated. Specifically, for any performance parameter data included in the performance indicator data, a corresponding parameter threshold can be set, so that when one or more performance parameter data exceeds its corresponding parameter threshold, a corresponding abnormal event is generated. For example, for the performance parameter data of CPU occupancy, the corresponding parameter threshold can be 80%, 90%, etc., which can be set according to actual needs. Therefore, during the actual test execution process, if the CPU occupancy of the test system exceeds the parameter threshold, it can be determined that the abnormal condition is met and a corresponding abnormal event is generated. The abnormal event can be, for example, that the CPU occupancy of the test system is too high.

[0081] For the running log data of the test case, corresponding log conditions can also be set, so that when the running log data corresponding to the test case meets the corresponding log conditions, a corresponding abnormal event can be generated. For example, the log condition can be set to: the running log data is incorrect, so that during the actual test execution process, if the running log data corresponding to a test case is incorrect, it can be determined that the abnormal condition is met and a corresponding abnormal event is generated. The abnormal event can be, for example,: the running log data is incorrect, etc. Alternatively, the running state of the test case can also be determined based on the running log data of the test case, so as to generate a corresponding abnormal event based on the running state, which will be explained in the subsequent embodiments.

[0082] In this embodiment, various monitoring tools are used to automatically monitor the operating data of the detection system during the test execution process to obtain the operating status of the test system, so that when the operating data meets abnormal conditions, corresponding abnormal events are generated to achieve automatic detection of abnormal events.

[0083] On this basis, in response to an abnormal event, a method for determining an abnormal type of the abnormal event may include:

[0084] For an abnormal event generated when the performance indicator data meets the first abnormal condition, determining that the abnormal type of the abnormal event is a performance failure;

[0085] When the running log data of the test case meets the second abnormal condition, it is determined that the abnormal type of the abnormal event is an execution failure.

[0086] Among them, the first abnormal condition and the second abnormal condition can be set according to actual test requirements.

[0087] Furthermore, according to the recovery mode corresponding to the exception type, the method of performing the corresponding fault handling operation may include:

[0088] Determine at least one target test case that meets the recovery condition according to the running states corresponding to the multiple test cases respectively;

[0089] When the exception type is a performance failure, switching at least one target test case to a backup system to re-execute at least one target test case in the backup system;

[0090] When the exception type is an execution failure, at least one target test case is re-executed.

[0091] Among them, at least one target test case that meets the recovery conditions may refer to one or more test cases that need to be re-executed. In practical applications, in response to an abnormal event, the execution of all test cases in the DAG structure can be interrupted, and the running status corresponding to all test cases can be obtained. The running status can be generated according to the running log data, and one or more test cases that need to be re-executed are determined based on the running status. For example, the target test case that needs to be re-executed can be a test case whose execution is interrupted due to an abnormal event, or a test case that fails to execute itself, etc.

[0092] After determining the target test case, the target test case can be re-executed in combination with the recovery method corresponding to the exception type of the exception event. Specifically, when the exception type is a performance failure, it indicates that the current test system no longer supports the continued execution of the test case, and at least one target test case can be switched to the backup system, and the at least one target test case can be re-executed in the backup system. Among them, the backup system refers to a backup system for the test system, and the performance data of the backup system is normal. The backup system can be specified in combination with the actual test task scheduling situation. When the exception type is an execution failure, the at least one target test case can be directly re-executed. In other words, if the performance data of the test system is normal, the at least one target test case can be directly re-executed without replacing the test system.

[0093] In this embodiment, the abnormal type of the abnormal event is determined according to different operating data of the detected test system in combination with different abnormal conditions set, and at least one target test case that meets the recovery conditions is determined according to the operating states corresponding to multiple test cases in the DAG structure. The at least one target test case is re-executed according to the recovery methods corresponding to the different abnormal types, thereby realizing automatic fault recovery during the test execution process and improving the fault recovery efficiency. In addition, different recovery methods are used for abnormal events of different abnormal types, thereby further improving the accuracy and rationality of fault recovery.

[0094] In some embodiments, the running states corresponding to the multiple test cases can be determined by the running log data. Therefore, the above method can also include:

[0095] Based on the operation log data corresponding to the multiple test cases, the operation status corresponding to the multiple test cases is determined. The operation status may include not started, in progress, success or failure. For example, for a certain test case, when there is no operation log data including the prompt information for starting execution, the operation status corresponding to the test case may be not started; when the operation log data is incorrect, the corresponding operation status may be failure; when the operation log data is correct and includes the execution result, the corresponding operation status may be success; when the operation log data is correct but does not include the execution result, the corresponding operation status may be in progress, and so on.

[0096] On this basis, according to the running states respectively corresponding to the multiple test cases, determining at least one target test case that meets the recovery condition may include:

[0097] Determine at least one first test case whose running status is failed or in progress, and based on a directed acyclic graph, determine at least one second test case that any first test case depends on; determine whether the test attributes of at least one first test case and at least one second test case meet the real-time requirements; based on the judgment result, determine at least one target test case that meets the recovery conditions.

[0098] Among them, meeting the real-time requirement may refer to having a real-time test requirement, which may be determined based on the test attributes. Whether the test attributes of at least one first test case and a second test case meet the real-time requirement may be judged respectively, and based on the judgment result, for example, the first test case and the second test case that meet the real-time requirement may be used as target test cases.

[0099] In order to improve the rationality of the target test case, optionally, based on the judgment result, determining at least one target test case that meets the recovery condition may include:

[0100] When the test attributes of the first test case meet the real-time requirements, the first test case, at least one second test case, and the previous test case on which the second test case that meets the real-time requirements depends are used as target test cases; otherwise, the first test case is used as the target test case.

[0101] Specifically, at least one first test case whose running status is failed or in progress can be determined, and based on the DAG structure, at least one second test case that any first test case depends on can be determined. Among them, the second test case that the first test case depends on can refer to the test case corresponding to the node in the DAG structure that has an edge relationship with the node corresponding to the first test case, and the direction of the edge points to the node corresponding to the first test case. There can be one or more second test cases. Figure 2 Taking the DAG structure shown as an example, if the first test case determined is test case D, according to the DAG structure, it can be determined that the second test cases that test case D depends on are test cases B and C.

[0102] Combined with the test properties of the first test case and the second test case, when the first test case meets the real-time requirement and the second test case does not meet the real-time requirement, the first test case and at least one second test case are used as target test cases; when the first test case meets the real-time requirement and there is a second test case that meets the real-time requirement, the first test case, at least one second test case, and the previous test case that the second test case that meets the real-time requirement depends on can be used as target test cases; when both the first test case and the second test case do not meet the real-time requirement, only the first test case needs to be used as the target test case. In other words, if the re-executed test case has a real-time test requirement, the test case and the test cases it depends on are re-executed to ensure the real-time nature of the input data obtained by the test case. Taking the DAG structure including multiple layers of dependencies as an example, such as test case m depends on test case n, test case n depends on test cases p and q, and test case q depends on test case x, if the test case m to be re-executed has a real-time testing requirement, then the test case n it depends on will be re-executed together. If the test case n it depends on also has a real-time testing requirement, then test cases p and q will continue to be re-executed together until a test case that does not have a real-time testing requirement is determined. If test case q does not have a real-time testing requirement, test case x will no longer be used as the target test case for re-execution.

[0103] by Figure 2 Taking the DAG structure shown as an example, if the first test case determined is test case D, according to the DAG structure, it can be determined that the second test cases that test case D depends on are test cases B and C. Combined with the test properties of test cases D, B and C, if test case D meets the real-time requirements, and test cases B and C do not meet the real-time requirements, test case B, test case C and test case D are taken as target test cases. If test case D meets the real-time requirements and test case B also meets the real-time requirements, test cases A~D are taken as target test cases. If test case D does not meet the requirements, only test case D can be taken as the target test case.

[0104] In this embodiment, after determining that at least one first test case has a running status of failure or in progress, at least one second test case on which the first test case depends is determined by combining the DAG structure, and combining the test attributes of the first test case and the second test case, such as whether real-time testing requirements are met, if the first test case meets the real-time requirements, the first test case, at least one dependent second test case, and the previous test case on which the second test case that meets the real-time requirements depends are all re-executed as target test cases; and if the first test case does not meet the real-time requirements, the first test case is directly re-executed as the target test case, thereby improving the rationality of determining the target test case and thereby improving the rationality of fault recovery.

[0105] In some embodiments, when the operation data meets the abnormal condition, generating the corresponding abnormal event may specifically be to generate the corresponding abnormal event in combination with the above-mentioned operation log data and the operation status.

[0106] As an optional implementation method, the running statuses corresponding to the multiple test cases can be determined based on the running log data corresponding to the multiple test cases, and when the running status of any test case is a failure, a corresponding abnormal event is generated.

[0107] As another optional implementation, based on the running log data corresponding to the multiple test cases, when it is determined that there are test cases that have failed to execute, a corresponding abnormal event is generated.

[0108] In order to facilitate smooth re-execution of the target test case, in some embodiments, in response to an abnormal event, the above method may further include:

[0109] For any test case, save the execution result of the previous test case that the test case depends on.

[0110] Therefore, the method of re-executing at least one target test case may include:

[0111] When the test attributes of the first test case do not meet the real-time requirements, the execution results corresponding to at least one second test case are used as input data of the first test case, and the first test case is re-executed; when the test attributes of the first test case meet the real-time requirements, according to the node execution order indicated by the directed acyclic graph, the execution result of the previous test case on which any target test case depends is used as input data of the target test case, and the target test case is re-executed, and the execution result obtained by re-executing the target test case is used as input data of a subsequent test case, and the subsequent test case is re-executed until the first test case is re-executed.

[0112] by Figure 2 Taking the DAG structure shown as an example, when the target test cases include test cases B~D, test cases B and C can be re-executed preferentially according to the node execution order indicated by the DAG. Since the execution result of test case A, on which test case B depends, is saved, the execution result can be directly used as the input data of test case B to re-execute test case B, and the execution results of test cases B and C can be used as the input data of test case D, and then test case D is executed to ensure the rationality of the re-execution.

[0113] In this embodiment, for any test case, by saving the execution result of the previous test case on which the test case depends, when re-executing the target test case, the execution result of the previous test case on which the target test case depends can be directly obtained and re-executed as input data to obtain the execution result, thereby ensuring smooth execution.

[0114] In some embodiments, using the execution result obtained by re-executing the target test case as input data for a subsequent test case may include:

[0115] Verify whether the execution result obtained by re-executing the target test case is consistent with the previous execution result corresponding to the target test case;

[0116] If the verification is consistent, the execution result obtained by re-executing the target test case is used as the input data of the next test case; otherwise, an exception prompt message is generated based on the verification result and sent to the tester corresponding to the object to be tested.

[0117] by Figure 2 Taking the DAG structure shown as an example, when the target test cases include test cases B~D, after re-executing test cases B and C, it can be verified whether the execution results obtained by re-executing test cases B and C are consistent with the previous execution results. When the verification is consistent, the execution results obtained by re-executing are used as input data for test case D, and test case D is re-executed, thereby ensuring that the test environment in which test case D is re-executed is consistent with the previous one, thereby improving the accuracy of fault recovery.

[0118] The abnormal prompt information may include the execution result obtained by re-executing the target test case, the previous corresponding execution result, and the comparison result between the two, etc. The abnormal prompt information may be used to prompt the tester to perform subsequent processing operations such as test case content detection or re-executing the target test case.

[0119] In this embodiment, by verifying whether the execution result obtained by re-executing the target test case is consistent with the previous corresponding execution result, it is ensured that the next test case is continued to be executed when the verification is consistent, thereby improving the accuracy of fault recovery.

[0120] In actual applications, if an exception still occurs after re-executing the target test case, you can re-execute the target test case again, or return to determine the exception type of the exception event, and follow the recovery method corresponding to the exception type to execute the corresponding fault handling operation steps to perform fault recovery again.

[0121] Optionally, a predetermined number of re-executions may be set. If the number of re-executions of the first test case is less than the predetermined number, the target test case may be re-executed. Otherwise, relevant personnel may be notified for processing.

[0122] In actual applications, when an abnormal event occurs during the test execution, a corresponding prompt message may be generated to notify relevant personnel. Therefore, in some embodiments, executing the corresponding fault handling operation may also include:

[0123] Generate fault prompt information based on abnormal events;

[0124] Send fault prompt information to relevant personnel.

[0125] The fault prompt information may include abnormal events, abnormal types, etc. Optionally, the fault prompt information may be sent to corresponding personnel according to the abnormal type. For example, when the abnormal type is a performance fault, the fault prompt information may be sent to the administrator of the test system; when the abnormal type is an execution fault, the fault prompt information may be sent to the tester of the object to be tested, and so on.

[0126] By generating fault prompt information based on abnormal events and sending the fault prompt information to relevant personnel, relevant personnel can be informed of the fault conditions during the test execution process, so as to prompt relevant personnel to perform subsequent processing operations such as repairing faulty machines and correcting erroneous programs.

[0127] Optionally, the above method may further include:

[0128] Based on the fault level conditions satisfied by the operating data of the test system, the fault level and the notification method corresponding to the fault level are determined; and the fault prompt information is sent to relevant personnel according to the notification method corresponding to the fault level.

[0129] For example, for the performance parameter data of CPU occupancy rate, you can set the occupancy rate to 80% as the first level fault, the occupancy rate to 90% as the second level fault, and so on. For different fault levels, you can also set different notification methods. The notification method can include the scope of notification personnel, notification channels, etc. For example, when the fault level is level 1, notification can be made through SMS, and when the fault level is level 2, notification can be made through multiple channels such as SMS, phone calls, and emails.

[0130] Optionally, the fault level can also be determined according to the abnormal type of the abnormal event. For example, for an abnormal event of an execution failure, the fault level can be set to level one, for an abnormal event of a performance failure, the fault level can be set to level two, and so on. The corresponding notification method can be, for example, when the fault level is level one, the tester of the object to be tested can be notified, when the fault level is level two, the management personnel of the test system and the tester of the object to be tested can be notified, and so on.

[0131] By setting different fault levels and notification methods corresponding to different fault levels, fault prompt information is sent to relevant personnel according to the notification method corresponding to the fault level of the abnormal event, thereby improving the flexibility and rationality of fault prompt information notification.

[0132] Optionally, a fault log may be generated in combination with the running log data of at least one target test case, and the fault log may be sent to relevant personnel.

[0133] Among them, the fault log may include abnormal operation data, abnormal events, abnormal types, recovery methods, determined target test cases to be re-executed, execution results obtained by re-execution, and the previous execution results, etc. Optionally, the fault log may be sent to corresponding personnel according to the type of abnormality. For example, when the abnormality type is a performance fault, the fault log may be sent to the administrator of the test system and the tester of the object to be tested. When the abnormality type is an execution fault, the fault log may be sent to the tester of the object to be tested, and so on.

[0134] By generating a fault log and sending it to relevant personnel, relevant personnel can be informed of the fault conditions during the test execution and clarify the fault recovery conditions.

[0135] In actual applications, the above test failure may also be caused by the update of the test case version. It is understandable that at this time, multiple test cases of the current test version may not be successfully executed. Therefore, after sending the fault prompt information to relevant personnel, the above method may also include:

[0136] According to the version recovery request triggered by relevant personnel, multiple historical test cases and historical directed acyclic graphs corresponding to the target test version are determined;

[0137] Execute multiple historical test cases according to the node execution order indicated by the historical directed acyclic graph.

[0138] The target test version may be any historical test version before the current test version is updated. The current test version and the target test version may be determined according to the test version identifier included in the description information of the test case configuration. Thus, the test is switched to the historical test case of the target test version before the update to perform the test and realize fault recovery.

[0139] like Figure 4 As shown, it is a structural schematic diagram of an embodiment of a testing device provided by the present application, and the device may include the following modules.

[0140] A first determination module 401 is used to determine multiple test cases of the object to be tested and the dependency relationship between the multiple test cases;

[0141] A construction module 402 is used to construct a directed acyclic graph using multiple test cases as nodes and dependencies between the multiple test cases as edges;

[0142] The testing module 403 is used to execute the test cases corresponding to the multiple nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested.

[0143] like Figure 5 As shown, it is a structural schematic diagram of another embodiment of a testing device provided by the present application, and the device may include the following modules.

[0144] A first determination module 401 is used to determine multiple test cases of the object to be tested and the dependency relationship between the multiple test cases;

[0145] A construction module 402 is used to construct a directed acyclic graph using multiple test cases as nodes and dependencies between the multiple test cases as edges;

[0146] The testing module 403 is used to execute the test cases corresponding to the multiple nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested;

[0147] A second determination module 504 is used to determine the abnormal type of the abnormal event in response to the abnormal event;

[0148] The fault handling module 505 is used to perform corresponding fault handling operations according to the recovery method corresponding to the exception type.

[0149] In some embodiments, the test module 403 may be specifically configured to execute test cases corresponding to a plurality of nodes respectively according to the node execution order indicated by the directed acyclic graph, and to execute test cases corresponding to at least two nodes having no dependency relationship respectively in parallel.

[0150] In some embodiments, the apparatus may further include:

[0151] A detection module is used to detect the running data of the test system during the execution of multiple test cases;

[0152] The first generating module is used to generate a corresponding abnormal event when the operating data meets the abnormal condition.

[0153] In some embodiments, the operation data may include performance indicator data corresponding to the test system, and / or operation log data of the test case;

[0154] The second determination module 504 may be specifically used to determine that the abnormal type of the abnormal event is a performance fault for the abnormal event generated when the performance indicator data meets the first abnormal condition; and to determine that the abnormal type of the abnormal event is an execution fault for the abnormal event generated when the running log data of the test case meets the second abnormal condition;

[0155] The fault handling module 505 can be specifically used to determine at least one target test case that meets the recovery conditions based on the running states corresponding to multiple test cases; when the exception type is a performance failure, switch at least one target test case to a backup system to re-execute at least one target test case in the backup system; when the exception type is an execution failure, re-execute at least one target test case.

[0156] In some embodiments, the apparatus may further include:

[0157] A second generating module is used to determine the running status corresponding to the plurality of test cases respectively based on the running log data corresponding to the plurality of test cases respectively; the running status may include not started, in progress, successful or failed;

[0158] A first acquisition module is used to obtain status identifiers corresponding to multiple test cases respectively;

[0159] The second determination module 504 can be specifically used to determine at least one first test case whose running status is failed or in progress; based on the directed acyclic graph, determine at least one second test case that any first test case depends on; determine whether the test attributes of at least one first test case and at least one second test case meet the real-time requirements; based on the judgment result, determine at least one target test case that meets the recovery conditions.

[0160] In some embodiments, the second determination 504 can be specifically used to use the first test case, at least one second test case, and the previous test case on which the second test case that meets the real-time requirement depends as target test cases when the test attributes of the first test case meet the real-time requirement; otherwise, use the first test case as the target test case.

[0161] In some embodiments, the apparatus may further include:

[0162] The saving module is used to save the execution result of the previous test case on which the test case depends for any test case;

[0163] The fault handling module 505 can be specifically used to use the execution result of at least one second test case as the input data of the first test case and re-execute the first test case when the test attribute of the first test case does not meet the real-time requirement; when the test attribute of the first test case meets the real-time requirement, according to the node execution order indicated by the directed acyclic graph, use the execution result of the previous test case on which any target test case depends as the input data of the target test case, re-execute the target test case, and use the execution result obtained by re-executing the target test case as the input data of the next test case, and re-execute the next test case until the first test case is re-executed.

[0164] In some embodiments, the fault handling module 505 may be specifically configured to re-execute at least one target test case when the number of re-executions of the first test case is less than a predetermined number.

[0165] In some embodiments, the first generation module can be specifically used to determine the running status corresponding to multiple test cases based on the running log data corresponding to the multiple test cases; the running status includes not started, in progress, successful or failed; when the running status of any test case is failure, a corresponding abnormal event is generated.

[0166] In some embodiments, the fault handling module 505 may be specifically configured to generate fault prompt information based on an abnormal event and send the fault prompt information to relevant personnel.

[0167] In some embodiments, the apparatus may further include:

[0168] A third determination module, used to determine the fault level and the notification method corresponding to the fault level based on the fault level conditions satisfied by the operation data;

[0169] The fault handling module 505 may be specifically configured to send fault prompt information to relevant personnel according to a notification method corresponding to the fault level.

[0170] In some embodiments, the apparatus may further include:

[0171] The notification module is used to generate a fault log in combination with the running log data of at least one target test case; and send the fault log to relevant personnel.

[0172] In some embodiments, the apparatus may further include:

[0173] The fourth determination module is used to determine multiple historical test cases and a historical directed acyclic graph corresponding to the target test version according to a version recovery request triggered by relevant personnel; and execute multiple historical test cases according to the node execution order indicated by the historical directed acyclic graph.

[0174] In some embodiments, the apparatus may further include:

[0175] A verification module is used to verify whether the execution result obtained by re-executing the target test case is consistent with the corresponding previous execution result;

[0176] The fault handling module 505 can be specifically used to use the execution result obtained by re-executing the target test case as the input data of the next test case when the verification is consistent; otherwise, generate abnormal prompt information based on the verification result and send the abnormal prompt information to the tester corresponding to the object to be tested.

[0177] In some embodiments, the apparatus may further include:

[0178] The second acquisition module is used to respond to the tester's configuration request and obtain the configuration information sent by the configuration end. The configuration information may include multiple test cases of the object to be tested, the dependency relationship between the multiple test cases, and the recovery methods corresponding to different exception types; the configuration request is generated by the configuration end in response to the tester's configuration operation.

[0179] In some embodiments, the apparatus may further include:

[0180] Configuration module, used to provide a configuration interface;

[0181] The third acquisition module is used to obtain configuration information in response to the configuration operation of the tester. The configuration information may include multiple test cases of the object to be tested, dependencies between the multiple test cases, and recovery methods corresponding to different exception types.

[0182] For ease of understanding, the following Figure 6 The system architecture diagram of the test system shown in the figure illustrates the technical solution of the present application. Figure 6As shown, the test system may include a test terminal 601 and a configuration terminal 602. The configuration terminal 602 may provide a configuration interface, in which a tester may configure multiple test cases of a to-be-tested object, dependencies between multiple test cases, test properties corresponding to multiple test cases, and recovery methods corresponding to different exception types, and other configuration information, and send the above configuration information to the test terminal 601.

[0183] The test end 601 can determine the multiple test cases of the object to be tested and the dependencies between the multiple test cases in response to the test request, use the multiple test cases as nodes, the dependencies between the multiple test cases as edges, construct a directed acyclic graph, and execute the test cases corresponding to the multiple nodes according to the node execution order indicated by the directed acyclic graph to test the object to be tested. Specifically, the test cases corresponding to the multiple nodes can be executed according to the node execution order indicated by the directed acyclic graph, and the test cases corresponding to at least two nodes in which there is no dependency are executed in parallel. Thus, the test method of executing multiple test cases one by one in the traditional scheme and then determining whether the object to be tested meets the specific requirements according to the execution results of the multiple test cases is solved, resulting in a large number of test cases, a long time consumption, and a low test problem, and it is possible to avoid the problems such as deadlock that may occur when executing multiple test cases at the same time, and realize efficient and orderly test case execution, and improve the rationality of the test.

[0184] During the test case execution process, the test end 601 can also respond to the abnormal event, determine the abnormal type of the abnormal event, and perform the corresponding fault handling operation according to the recovery method corresponding to the abnormal type. Fault handling is achieved without manual intervention, ensuring the efficiency and accuracy of fault handling, further improving test efficiency, and improving test rationality.

[0185] Optionally, the test end 601 can also detect the running data of the test system using performance monitoring tools, log monitoring systems, and health check programs during the execution of multiple test cases, such as using performance monitoring tools to detect the performance indicator data of the test system, using log monitoring systems to detect the running log data of the test cases, and generating corresponding abnormal events when the running data meets abnormal conditions. By using various monitoring tools to automatically monitor the running data of the detection system during the test execution process, the running status of the test system is known, and when the running data meets abnormal conditions, corresponding abnormal events are generated to achieve automatic detection of abnormal events.

[0186] Optionally, the test end 601 can also determine that the abnormal type of the abnormal event is a performance fault for the abnormal event generated when the performance indicator data meets the first abnormal condition; and determine that the abnormal type of the abnormal event is an execution fault for the running log data of the test case meets the second abnormal condition. In addition, according to the running states corresponding to the multiple test cases, at least one target test case that meets the recovery condition is determined; when the abnormal type is a performance fault, at least one target test case is switched to the backup system to re-execute the at least one target test case in the backup system; when the abnormal type is an execution fault, at least one target test case is re-executed. By determining the abnormal type of the abnormal event according to the different running data of the detected test system and the different abnormal conditions set, and determining at least one target test case that meets the recovery condition according to the running states corresponding to the multiple test cases in the DAG structure, re-execute the at least one target test case according to the recovery methods corresponding to the different abnormal types, automatic fault recovery is achieved during the test execution process, and the fault recovery efficiency is improved. Different recovery methods are used for abnormal events of different abnormal types, and the accuracy and rationality of fault recovery are further improved.

[0187] Optionally, the test terminal 601 can also determine the running statuses corresponding to the multiple test cases based on the running log data corresponding to the multiple test cases, determine at least one first test case whose running status is failed or in progress, and determine at least one second test case that any first test case depends on based on the directed acyclic graph; if the test attribute of the first test case meets the real-time requirement, the first test case, at least one second test case, and the previous test case that the second test case that meets the real-time requirement depends on are used as target test cases; otherwise, the first test case is used as the target test case. This improves the rationality of determining the target test case, and further improves the rationality of fault recovery.

[0188] Optionally, the test end 601 can also determine the running status corresponding to multiple test cases based on the running log data corresponding to the multiple test cases, and generate a corresponding exception event when the running status of any test case is a failure; or generate a corresponding exception event when it is determined that there is a test case that failed to execute based on the running log data corresponding to the multiple test cases.

[0189] Optionally, after responding to an abnormal event, the test end 601 can also save the execution result of the previous test case on which the test case depends for any test case. When re-executing the target test case, the execution result of the previous test case on which the target test case depends can be directly obtained and re-executed as input data to obtain the execution result, thereby ensuring smooth execution.

[0190] Optionally, the test end 601 may also verify whether the execution result obtained by re-executing the target test case is consistent with the previous corresponding execution result, and ensure that the next test case is continued to be executed when the verification is consistent, thereby improving the accuracy of fault recovery.

[0191] Optionally, the test end 601 can also generate fault prompt information based on abnormal events, and send the fault prompt information to relevant personnel, so that the relevant personnel can be informed of the fault conditions during the test execution process, so as to prompt the relevant personnel to perform subsequent processing operations such as repairing the faulty machine and correcting the erroneous program.

[0192] Optionally, the test end 601 can also determine the fault level and the notification method corresponding to the fault level based on the fault level conditions satisfied by the operating data; and send the fault prompt information to relevant personnel according to the notification method corresponding to the fault level, thereby improving the flexibility and rationality of the fault prompt information notification.

[0193] Optionally, the test end 601 may also generate a fault log in combination with the running log data of at least one target test case; and send the fault log to relevant personnel so that the relevant personnel can be informed of the fault conditions during the test execution and clarify the fault recovery conditions.

[0194] Of course, in another optional implementation, the test system may only include a test end, and the test end itself may provide a configuration interface, in which the tester may configure information, and in response to a test request, determine multiple test cases of the object to be tested and the dependencies between the multiple test cases, and construct a directed acyclic graph with multiple test cases as nodes and the dependencies between the multiple test cases as edges, and execute the test cases corresponding to the multiple nodes in the order of node execution indicated by the directed acyclic graph to test the object to be tested. And during the test execution process, in response to an abnormal event, determine the abnormal type of the abnormal event, and perform the corresponding fault handling operation according to the recovery method corresponding to the abnormal type. The specific implementation process will not be repeated.

[0195] like Figure 7 , which is a schematic diagram of a structure of an embodiment of a computing device provided by the present application, the device may include a storage component 701 and a processing component 702;

[0196] The storage component 701 can be used to store one or more computer program instructions, wherein the one or more computer program instructions are called and executed by the processing component 702 to implement Figure 1 or Figure 3 Test method shown.

[0197] Of course, the above computing device may also include other components, such as input / output interfaces, communication components, etc.

[0198] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices.

[0199] It should be noted that the above computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. It can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device.

[0200] Optionally, the computing device may also be implemented as an electronic device. An electronic device may refer to a device used by a user and having the functions of surfing the Internet, computing, communicating, etc. required by the user, such as a mobile phone, a tablet computer, a personal computer, a wearable device, etc. It is understandable that the electronic device may also include a display component, an input / output interface, a communication component, and other components, which will not be described in detail.

[0201] In one or more of the above embodiments, the processing component may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0202] The storage component is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0203] The display component may be an electroluminescent (EL) element, a liquid crystal display or a micro display having a similar structure, or a retinal direct display or a similar laser scanning display.

[0204] The present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a computer, Figure 1 or Figure 3The computer readable medium may be included in the computing device described in the above embodiment; or it may exist independently without being assembled into the computing device.

[0205] The computer-readable storage medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above.

[0206] The present application also provides a computer program product, which includes a computer program carried on a computer-readable storage medium, and when the computer program is executed by a computer, Figure 1 or Figure 3 Test method shown.

[0207] In such an embodiment, the computer program may be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, various functions defined in the system of the present application are performed.

[0208] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0209] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0210] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A testing method, characterized in that: include: Determine multiple test cases for the object to be tested and dependencies between the multiple test cases; Constructing a directed acyclic graph by using the multiple test cases as nodes and the dependency relationships between the multiple test cases as edges; Execute the test cases corresponding to the multiple nodes respectively according to the node execution order indicated by the directed acyclic graph to test the object to be tested; In response to an abnormal event, determining an abnormal type of the abnormal event; Perform corresponding fault handling operations according to the recovery method corresponding to the exception type; Also includes: During the execution of the plurality of test cases, detecting the operation data of the test system; the operation data includes the operation log data of the test cases; Determine the running statuses respectively corresponding to the multiple test cases based on the running log data respectively corresponding to the multiple test cases; The performing of corresponding fault handling operations according to the recovery mode corresponding to the exception type includes: Determine at least one first test case whose running status is failed or in progress; Based on the directed acyclic graph, determining at least one second test case on which any first test case depends; Determine whether the test attributes of the at least one first test case and the at least one second test case meet the real-time requirements; the test attributes include at least one of the test content, test function, required test resources, test machine, execution time and execution priority, the real-time requirement includes the requirement of real-time testing, and the real-time requirement is determined according to the test attributes; Based on the judgment result, determining at least one target test case that meets the recovery condition; Re-execute the at least one target test case according to the recovery method corresponding to the exception type.

2. The method according to claim 1, characterized in that The executing test cases respectively corresponding to the plurality of nodes according to the node execution order indicated by the directed acyclic graph to test the object to be tested comprises: The test cases corresponding to the multiple nodes are executed according to the node execution order indicated by the directed acyclic graph, and the test cases corresponding to at least two nodes without dependency relationship are executed in parallel.

3. The method according to claim 1, characterized in that: Also includes: When the operating data meets the abnormal condition, a corresponding abnormal event is generated.

4. The method according to claim 1, characterized in that The operation data includes performance indicator data corresponding to the test system; In response to the abnormal event, determining the abnormal type of the abnormal event includes: For an abnormal event generated when the performance indicator data meets the first abnormal condition, determining that the abnormal type of the abnormal event is a performance failure; For an abnormal event generated when the running log data of the test case meets the second abnormal condition, determining that the abnormal type of the abnormal event is an execution failure; The performing of corresponding fault handling operations according to the recovery mode corresponding to the exception type includes: In the case where the abnormality type is a performance failure, switching the at least one target test case to a standby system to re-execute the at least one target test case in the standby system; When the exception type is an execution failure, the at least one target test case is re-executed.

5. The method according to claim 1, characterized in that The operation status includes not started, in progress, successful or failed.

6. The method according to claim 1, characterized in that The determining, based on the judgment result, at least one target test case that meets the recovery condition comprises: In the case where the test attribute of the first test case meets the real-time requirement, the first test case, the at least one second test case, and a previous test case on which the second test case meeting the real-time requirement depends are used as target test cases; Otherwise, the first test case is used as the target test case.

7. The method according to claim 6, characterized in that Also includes: For any test case, save the execution result of the previous test case on which the test case depends; Re-executing the at least one target test case comprises: In the case where the test attribute of the first test case does not meet the real-time requirement, using the execution result of the at least one second test case as input data of the first test case, and re-executing the first test case; When the test attributes of the first test case meet the real-time requirements, the execution result of the previous test case on which any target test case depends is used as input data of the target test case, and the target test case is re-executed in accordance with the node execution order indicated by the directed acyclic graph. The execution result obtained by re-executing the target test case is used as input data of a subsequent test case, and the subsequent test case is re-executed until the first test case is re-executed.

8. The method according to claim 1, characterized in that: The performing of the corresponding fault handling operation includes: Generate fault prompt information based on the abnormal event; The fault prompt information is sent to relevant personnel.

9. The method according to claim 8, characterized in that Also includes: According to the version recovery request triggered by the relevant personnel, multiple historical test cases and a historical directed acyclic graph corresponding to the target test version are determined; The multiple historical test cases are executed according to the node execution order indicated by the historical directed acyclic graph.

10. The method according to claim 1, characterized in that Also includes: In response to a configuration request from a tester, configuration information is obtained, wherein the configuration information includes multiple test cases for the object to be tested, dependencies between the multiple test cases, and recovery methods corresponding to different exception types; the configuration request is generated by the configuration end in response to the configuration operation of the tester.

11. The method according to claim 1, characterized in that Also includes: Provide a configuration interface; In response to the configuration operation of the tester, configuration information is obtained, where the configuration information includes a plurality of test cases of the object to be tested, dependencies between the plurality of test cases, and recovery methods corresponding to different exception types.

12. A computing device, characterized in that: It comprises a storage component and a processing component; the storage component stores one or more computer program instructions, the computer program instructions are called and executed by the processing component, and the processing component executes the one or more computer program instructions to implement the test method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that: A computer program is stored, and the computer program is executed by a computer to implement the testing method according to any one of claims 1 to 11.

14. A computer program product, characterized in that A computer program is stored, and when the computer program is executed by a computer, the test method according to any one of claims 1 to 11 is implemented.

Images