Test case execution method, electronic equipment, storage medium and program product

Abstract

The invention discloses a test case execution method, electronic equipment, a storage medium and a program product, and relates to the technical field of testing, the explicit dependency relationship among test cases is identified according to a test case set, and the implicit dependency relationship among the test cases is identified according to historical execution data; according to the identification result of the explicit dependency relationship and the implicit dependency relationship, a dependency graph of the test case set is constructed, the dependency graph is constructed in a mode of combining the static test case set and the dynamic historical execution data, so that the constructed dependency graph is more accurate, and then the dependency type of the test case is determined according to the dependency graph. The execution sequence of the test cases in the test case set is determined according to the dependency types, so that different characteristics of the test cases of different dependency types are considered, the test efficiency is improved, tests are executed according to the execution sequence, invalid tests are prevented from being executed, the test time is shortened, and the test cost is reduced; the technical problems of low test efficiency and high test cost in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to a test case execution method, electronic equipment, storage medium, and program product. Background Art

[0002] As software systems continue to iterate and evolve, the test suites for modern software products are becoming increasingly large and complex. The exponential growth in the number of test suites is leading to increasing test execution costs. Improving test efficiency and reducing test costs are key issues that the testing industry must address.

[0003] Related technologies usually use fixed order or random scheduling to execute tests. Among them, the method based on fixed order mainly relies on static analysis to generate an execution order that cannot be dynamically adjusted. However, static analysis is not accurate enough in identifying relationships, which will lead to the execution of some invalid test cases, thereby resulting in low test efficiency; the method based on random scheduling to execute tests may have a high-priority test case that depends on the execution result of a low-priority test case, and the high-priority test case will be executed first, resulting in low test efficiency and high test cost. Summary of the Invention

[0004] The present application provides a test case execution method, electronic device, storage medium and program product to at least solve the problems of low testing efficiency and high testing cost in related technologies.

[0005] The present application provides a test case execution method, comprising the following steps: obtaining a test case set and historical execution data of the test cases; identifying explicit dependencies between test cases based on the test case set; identifying implicit dependencies between test cases based on the historical execution data; constructing a dependency graph of the test case set based on the identification results of the explicit dependencies and the implicit dependencies, wherein the dependency graph includes directed edges and nodes, the nodes represent test cases, and the directed edges represent the existence of dependencies between the test cases; determining the dependency types of the test cases in the test case set based on the dependency graph, determining the execution order of the test cases in the test case set based on the dependency types, and executing the test cases in the test case set according to the execution order.

[0006] The present application also provides a test case execution device, including: an acquisition module for acquiring a test case set and historical execution data of the test cases; a first identification module for identifying explicit dependencies between test cases based on the test case set; a second identification module for identifying implicit dependencies between test cases based on the historical execution data; a construction module for constructing a dependency graph of the test case set based on the identification results of explicit dependencies and implicit dependencies, wherein the dependency graph includes directed edges and nodes, the nodes represent test cases, and the directed edges represent the existence of dependencies between test cases; an execution module for determining the dependency types of the test cases in the test case set based on the dependency graph, determining the execution order of the test cases in the test case set based on the dependency types, and executing the test cases in the test case set according to the execution order.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned test case execution methods when executing the computer program.

[0008] The present application also provides a non-volatile computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned test case execution methods are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned test case execution methods when executed by a processor.

[0010] Through this application, explicit dependencies between test cases can be identified based on the test case set, implicit dependencies between test cases can be identified based on historical execution data, and a dependency graph of the test case set can be constructed based on the identification results of explicit dependencies and implicit dependencies. The dependency graph can be constructed by combining a static test case set with dynamic historical execution data to make the constructed dependency graph more accurate, and then the dependency type of the test case can be determined based on the dependency graph, and the execution order of the test cases in the test case set can be determined based on the dependency type, so as to take into account the different characteristics of test cases with different dependency types and improve the test efficiency. Then, the test can be executed in the execution order to avoid executing invalid tests, which can shorten the test time and reduce the test cost to a certain extent. Therefore, the technical problems of low test efficiency and high test cost in related technologies can be solved, and the technical effect of improving test efficiency and reducing test cost can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A flowchart of a test case execution method provided according to an embodiment of the present application; Figure 2 A complete flow chart of a test case execution method provided according to one embodiment of the present application; Figure 3 A schematic diagram of the architecture of a test case system provided according to an embodiment of the present application; Figure 4 A schematic diagram of a test case device provided according to an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0015] Before describing the solution of this application, the relevant technologies of this application are first explained to assist in understanding the solution of this application.

[0016] In related technologies, test execution scheduling methods often use fixed order or random scheduling. In the fixed order test execution method, the fixed order is determined based on static rules. For example, the execution sequence is generated based on manually preset fixed priorities (such as P0>P1>P2) (for example, core function testing is a high priority), and use cases are executed in a fixed order. Alternatively, the execution order is determined based on static dependencies, that is, scheduling based on topological sorting, which models use case dependencies as a directed acyclic graph and generates a linear execution sequence through topological sorting. The above test execution methods have the following problems: 1. Insufficient identification of dynamic dependencies: Test cases may have data dependencies (e.g., users needing to submit tasks that occupy resources), environment dependencies (e.g., configuration initialization), and state dependencies (e.g., user login status). If these dependencies are not explicitly declared, they can easily lead to execution failures, wasting time and resources. In other words, runtime dependencies cannot be captured, for example, a high-priority test case may depend on the initialization operations of a low-priority test case.

[0017] 2. Increased cost of repeated execution: When dependency chains are not identified, the same environment initialization operation may be executed independently by multiple use cases, extending the test execution time.

[0018] 3. Low resource utilization: In a multi-threaded / distributed environment, parallel execution of high-resource-consuming use cases can easily lead to memory overflow or CPU overload. In other words, resource requirements are ignored during parallel scheduling, resulting in resource conflicts or idle waste.

[0019] 4. Lack of real-time feedback and poor adaptability to dynamic scenarios: Static scheduling strategies cannot dynamically adjust subsequent plans based on execution results (such as use case failure). When a use case on the critical path fails, the static queue still executes all its subsequent use cases, resulting in redundant execution and increasing invalid testing costs.

[0020] 5. Not taking into account test characteristics: execution time, resource requirements, and priority.

[0021] To this end, the present application provides a test case execution method to solve at least one of the above technical problems.

[0022] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0023] An embodiment of the present application provides a test case execution method, and the method is described in detail in conjunction with the execution process of the test case execution method.

[0024] Figure 1 The present invention provides a flowchart of a test case execution method according to an embodiment of the present application.

[0025] like Figure 1As shown, the test case execution method includes the following steps: In step S101 , a test case set and historical execution data of the test cases are obtained.

[0026] Among them, the test case set includes text test cases and automated test cases. Test cases are a set of standardized test processes designed to verify product functions, performance, security and other characteristics during the development and testing of products (such as software). Text test cases are manual test bases written in natural language, tables or documents. Automated test cases are automatically executable scripts written in programming languages ​​or automation tools.

[0027] Historical execution data includes historical execution logs and historical data. Historical execution logs are detailed execution information automatically recorded during multiple past executions of test cases, including basic execution information, execution process data (including the underlying resources involved in the operation (such as accessed database tables, called interfaces, etc.)), etc. Historical data includes execution results and corresponding blocking reasons, bug numbers, test case execution time, etc.

[0028] In step S102, explicit dependencies between test cases are identified based on the test case set.

[0029] Among them, explicit dependencies are dependencies between test cases that are explicitly declared or can be directly read from the test case content.

[0030] In some embodiments of the present application, identifying explicit dependencies between test cases based on a test case set includes: reading at least one of the prerequisites, test steps, and expected results in a text test case, and determining the explicit dependencies between the text test cases based on at least one of the prerequisites, test steps, and expected results; identifying function code dependency data of automated test cases, and determining the explicit dependencies between automated test cases based on the function code dependency data.

[0031] It is understandable that the embodiment of the present application can determine how to determine the explicit dependency relationship between corresponding test cases according to the type of test cases in the test case set. Specifically, 1. Text test cases.

[0032] At least one of a precondition, a test step, and an expected result in a text test case is read, and an explicit dependency relationship between the text test cases is determined based on at least one of the precondition, the test step, and the expected result.

[0033] For example, if the prerequisite of test case B explicitly states that test case A must be executed first and ensured to be executed successfully, then B depends on A (A→B). If the test steps of test case C include test data A generated by test case D, then C depends on D (D→C). The expected result of test case E needs to be compared with the execution result of test case F. For example, the output of E should be consistent with the output of F, so E depends on F (F→E).

[0034] 2. Automated test cases.

[0035] Identify the function code dependency data of automated test cases, and determine the explicit dependency relationships between automated test cases based on the function code dependency data.

[0036] The code2flow tool can be used to identify the function code dependencies of automated test cases and indirectly obtain the dependencies between test cases.

[0037] For example, if the code of automated test case B directly calls the return result of test case A, then B depends on A (A→B); The initialization function of test case C depends on the environment configuration function of test case script D, so C depends on D (D→C).

[0038] In step S103 , implicit dependencies between test cases are identified based on historical execution data.

[0039] Among them, implicit dependencies are potential dependencies that are not explicitly declared between test cases but are formed during the execution process due to shared resources, data interaction, etc.

[0040] In some embodiments of the present application, implicit dependencies between test cases are identified based on historical execution data, including: identifying at least one of the shared database tables and resource occupancy of the test cases in the historical execution log; and determining the implicit dependencies between test cases based on at least one of the shared database tables and resource occupancy.

[0041] It can be understood that the embodiments of the present application can identify at least one of the shared database tables and resource occupancy of test cases in historical execution logs, and determine the implicit dependencies between test cases based on at least one of the shared database tables and resource occupancy, thereby identifying the dependencies that are not explicitly declared between test cases and improving the accuracy of subsequent dependency graph construction.

[0042] For example, resource usage information recorded in historical execution logs shows that multiple test cases use the same resource, which may cause conflicts or data interference. For example, test case A and test case B frequently access the database table user_info at the same time during historical execution. A modifies the data in the table, and B reads the data (the result of B is correct only after A is executed). In this case, B depends on A (A→B).

[0043] In step S104, based on the identification results of explicit dependencies and implicit dependencies, a dependency graph of the test case set is constructed, wherein the dependency graph includes directed edges and nodes, the nodes represent test cases, and the directed edges represent the existence of dependencies between test cases.

[0044] It can be understood that the embodiments of the present application can construct a dependency graph of the test case set based on the identification results of explicit dependencies and implicit dependencies, and construct the dependency graph by combining a static test case set with dynamic historical execution data, so that the constructed dependency graph is more accurate, thereby improving the efficiency of subsequent testing, avoiding the execution of invalid tests, and shortening the testing time to a certain extent.

[0045] A dependency graph is a structured representation used to visualize or model the dependencies between various elements in a system (such as components, tasks, data, and services). It is presented as a graph, with nodes representing system elements and edges (connections) representing the dependencies between elements, helping users understand the complex interaction logic and processes within the system.

[0046] In some embodiments of the present application, a dependency graph of a test case set is constructed based on the identification results of explicit dependencies and implicit dependencies, including: determining the existence of dependencies between test cases based on the identification results of explicit dependencies and implicit dependencies; using test cases as nodes, if there are dependencies between test cases, generating directed edges between the nodes corresponding to the test cases; if there are no dependencies between the test cases, generating nodes corresponding to the test cases.

[0047] It can be understood that the embodiment of the present application can determine whether there is a dependency relationship between test cases based on the identification results of explicit dependencies and implicit dependencies, and use the test cases as nodes in the dependency graph. When there is a dependency relationship between test cases, directed edges are generated between the nodes corresponding to the test cases. If there is no dependency relationship between test cases, the node corresponding to the test case is an independent node in the dependency graph.

[0048] Specifically, the dependency graph includes directed edges and nodes. Nodes represent test cases. For test cases with dependency relationships, the corresponding nodes are connected by directed edges. For example, node A points to node B, indicating that B depends on A. If there is no dependency relationship, the corresponding node is an isolated node.

[0049] In step S105 , the dependency types of the test cases in the test case set are determined according to the dependency graph, the execution order of the test cases in the test case set is determined according to the dependency types, and the test cases in the test case set are executed according to the execution order.

[0050] The dependency type includes a first type and a second type. The first type is a type with a dependency relationship, and the second type is a type without a dependency relationship.

[0051] It can be understood that the embodiments of the present application can determine the dependency type of the test case based on the dependency graph, and determine the execution order of the test cases in the test case set based on the dependency type, so as to take into account the different characteristics of test cases with different dependency types, improve the test efficiency, and then execute the test in the execution order.

[0052] In some embodiments of the present application, before determining the dependency type of the test cases in the test case set according to the dependency graph, it also includes: identifying the execution results and problem data of the test cases in the historical data; and adjusting the weights of the directed edges based on the execution results and problem data.

[0053] Among them, the problem data includes the blocking reason, BUG number, etc.; the weight of the directed edge includes the degree of dependency (strong, weak), resource requirements (CPU / accelerator card), and execution time (the time required from starting to execute the test case to completing all steps of this test case).

[0054] It can be understood that the embodiments of the present application can identify the execution results and problem data of test cases in historical data, and adjust the weights of directed edges in the dependency graph based on the execution results and problem data to improve the rationality of subsequent execution sequence planning and improve testing efficiency.

[0055] Specifically, the embodiments of the present application can dynamically adjust the dependency strength in combination with historical data, for example, frequently failing dependencies can be marked as "strong"; historical data includes the execution logs of historical versions, test case execution results, and the corresponding blocking reasons, BUG numbers, and test case execution times. In historical versions, if the test case execution result is PASS, it means that the test case has passed; if the test case execution result is FAIL, it means that the test case has failed, and the corresponding BUG number needs to be associated; if the test case execution result is BLOCK, it means that the test case has been blocked, and the reason for the blocked test needs to be noted.

[0056] Test case execution results reveal inter-test case dependencies and their strength. For example, if a test case fails in historical execution and one or more other test cases often fail at the same time, this indicates a dependency between this test case and the test cases that frequently fail at the same time, and this dependency can be adjusted. If a test case fails with BUG 123 associated, and one or more other test cases are simultaneously blocked due to BUG 123, this indicates a strong dependency between this test case and the other blocked test cases, and this dependency can be adjusted. The execution time of each test case is displayed, which can be used to optimize the execution sequence.

[0057] In addition, the historical execution log records the resource requirements of each use case, including CPU and accelerator cards, which can also be used to adjust edge weights.

[0058] In some embodiments of the present application, the dependency type of a test case in a test case set is determined based on a dependency graph, including: identifying directed edges and nodes in the dependency graph; if there is a directed edge at a node in the dependency graph, determining that the dependency type of the test case corresponding to the node is the first type; if there is no directed edge at a node in the dependency graph, determining that the dependency type of the test case corresponding to the node is the second type.

[0059] It can be understood that in the embodiment of the present application, when there is a directed edge between nodes in the dependency graph, the dependency type of the test case corresponding to the node is determined to be the first type. If there is no directed edge between the nodes, the dependency type of the test case corresponding to the node is determined to be the second type.

[0060] For example, in the dependency graph, there is a directed edge between node A and node B, and node C is a single node, which means that the dependency type between the test cases corresponding to node B and node A is the first type, and the dependency type of node C is the second type.

[0061] In some embodiments of the present application, the execution order of test cases in a test case set is determined according to the dependency type, including: obtaining the priority of the test case; identifying at least one characteristic data in the weight of the directed edge between the test cases, wherein the characteristic data includes the dependency degree of the test case, the resource requirement of the test case and the execution time of the test case; based on the dependency type, the priority of the test case and at least one characteristic data of the test case, determining the execution order of the test cases in the test case set.

[0062] Among them, the initial priority of the test case is calibrated in advance.

[0063] It can be understood that the embodiments of the present application can determine the execution data of the test cases in the test case set based on the dependency type, the priority of the test case and at least one characteristic data of the test case, so as to determine the execution order of different types of test cases.

[0064] In some embodiments of the present application, the execution order of test cases in a test case set is determined based on the dependency type, the priority of the test case and at least one characteristic data in the test case, including: if the dependency type of the test case is the first type, the execution order of the test case is determined based on the priority of the test case, the dependency degree of the test case, and the execution time of the test case; if the dependency type of the test case is the second type, the execution order of the test case is determined based on the priority of the test case and the execution time of the test case.

[0065] It can be understood that in the embodiment of the present application, when the dependency type of the test case is the first type, that is, there is a dependency relationship between the test cases, the execution order of the test cases is determined based on the priority of the test case, the degree of dependency of the test case, and the execution time of the test case. When the dependency type of the test case is the second type, that is, there is no dependency relationship between the test cases, the execution order of the test cases only needs to be determined based on the priority of the test case and the execution time of the test case.

[0066] In addition, it should be noted that when the dependency type of the test case is the second type, the execution time of the test case can be directly obtained.

[0067] In some embodiments of the present application, the execution order of test cases is determined based on the priority of the test cases, the dependency level of the test cases, and the execution time of the test cases, including: sorting the test cases from high to low based on the priority of the test cases, and adjusting the execution order of the test cases based on the dependency level of the test cases and the execution time of the test cases.

[0068] It can be understood that the embodiment of the present application can sort the test cases from high to low based on the priority of the test cases when the dependency type of the test cases is the first type, and adjust the execution order of the test cases based on the dependency degree of the test cases and the execution time of the test cases, so as to maximize the test efficiency and shorten the test time.

[0069] For example, the priority of test case A is P0 (highest), and the execution time is 10 minutes; the priority of test case B is P0, and B depends on A with a strong degree of dependence (for example, it can only be executed after A succeeds), and the execution time is 5 minutes; the priority of test case C is P1 (medium), C depends on A with a weak degree of dependence (for example, execution can be attempted if A fails), and the execution time is 8 minutes; the priority of test case D is P1 (medium), D depends on B with a strong degree of dependence, and the execution time is 6 minutes.

[0070] The basic topology order is A → [B, C] → D. B has a higher priority than C. The order is adjusted to A → B → C. B takes less time to execute than C, and B is a strong dependency (affecting the execution of D). Prioritizing B allows D to start earlier. C is a weak dependency and has a lower priority. It is executed after B (does not affect the core process). The final execution order is A → B → D → C.

[0071] In some embodiments of the present application, the execution order of test cases is determined based on the priority of the test cases and the execution time of the test cases, including: determining the execution order of test cases based on a multi-objective optimization algorithm, the priority of the test cases and the execution time of the test cases, wherein the multi-objective optimization algorithm includes a first goal and a second goal, wherein the first goal is to maximize the sum of the priorities of multiple test cases, and the second goal is to minimize the sum of the execution time of multiple test cases.

[0072] The multi-objective optimization algorithm may be the NSGA-II algorithm.

[0073] It can be understood that the embodiments of the present application can determine the execution order of test cases based on the multi-objective optimization algorithm, the priority of the test cases and the execution time of the test cases. Specifically, the test cases are sorted according to the multi-objective optimization algorithm NSGA-II, the decision variable is the execution order of the test cases, the goal 1 (maximization) is the sum of the priorities (high priority use cases are ranked first), and the goal 2 (minimization) is the total execution time (short time use cases are prioritized to reduce blocking), and the constraint is that each use case can only be executed once.

[0074] This application can adopt a permutation coding method, such as [3,1,2] means executing use case 3 first, then use case 1, and finally use case 2, where the optimization factors are use case priority and execution time to maximize resource utilization.

[0075] In some embodiments of the present application, test cases in a test case set are executed in an execution order, including: executing the first type of test cases based on the execution order of test cases with a dependency type of the first type; and executing the second type of test cases based on the execution order of test cases with a dependency type of the second type.

[0076] Since in some cases, test cases need to be tested in multiple test environments, the embodiments of the present application can assign different types of test cases to testers in different test environments. The testers execute the corresponding test cases in sequence according to the specific execution order, thereby improving test efficiency.

[0077] In some embodiments of the present application, executing the second type of test cases based on the execution order of the second type of test cases includes: grouping the second type of test cases; allocating resources to the multiple groups of test cases based on the resource requirements of the multiple groups of test cases, wherein the multiple groups of test cases are executed in parallel.

[0078] It can be understood that the embodiment of the present application can group the second type of test cases and allocate resources to multiple groups of test cases based on their resource requirements to avoid resources such as the CPU (Central Processing Unit) and accelerator card exceeding the environmental upper limit, resulting in execution blockage. Multiple groups of test cases are executed in parallel to improve testing efficiency.

[0079] In some embodiments of the present application, after executing the test cases in the test case set in the execution order, it also includes: obtaining the execution result of the test case currently being executed; if the execution result is execution failure or execution blocking, and the dependency type of the test case currently being executed is the first type, then determining the downstream test cases of the test case currently being executed based on the dependency relationship, and not executing the downstream test cases of the test case currently being executed.

[0080] It can be understood that after executing the test cases in the test case set in the execution order, the embodiment of the present application can obtain the execution result of the test case currently being executed, and when it is the first type, the downstream test cases of the test case currently being executed are determined based on the dependency relationship, and the downstream test cases of the test case currently being executed are not executed, so as to avoid the downstream test cases from executing invalid tests, reduce resource consumption, and reduce testing costs.

[0081] In some embodiments of the present application, after executing the test cases in the test case set in the execution order, it also includes: identifying the execution time of the test case; if there is a target test case whose execution time exceeds the preset time, determining the downstream test cases related to the target test case based on the dependency relationship, and reducing the priority of the downstream test cases related to the target test case.

[0082] The preset time can be set according to specific circumstances and is not specifically limited thereto, for example, it can be set to 15 minutes or 10 minutes.

[0083] It can be understood that the embodiment of the present application can be the execution time of the test case. If there is a target test case whose execution time exceeds the preset time, the downstream test cases related to the target test case are determined based on the dependency relationship, and the priority of the downstream test cases related to the target test case is reduced. The execution order can then be dynamically adjusted to improve the test efficiency.

[0084] In some embodiments of the present application, before determining the execution order of test cases in a test case set based on the dependency type, the priority of the test case and at least one characteristic data in the test case, it also includes: if the dependency type of the test case is the first type, and the test case forms a directed loop between corresponding nodes in the dependency graph, then the multiple test cases in the directed loop are split into multiple sub-test cases.

[0085] In the related art, when a directed loop appears in a test case, manual intervention is required to stop the test loop. In the embodiment of the present application, when a directed loop exists in the dependency graph, multiple test cases in the directed loop are automatically split into multiple sub-test cases for execution, so as to avoid the cyclic execution of multiple test cases in the directed loop and avoid meaningless test loops.

[0086] For example, if there is a directed loop A→B→C→A in the dependency graph, it is automatically decomposed into atomic tasks A, B, and C, and divided into three subtasks (i.e., sub-test cases).

[0087] The following describes a method for executing a test case in an embodiment of the present application through a specific embodiment.

[0088] This application uses the analysis of test cases and test case execution logs to obtain the dependency graph of test cases, and formulate the test case execution order based on topological sorting, and monitor the execution results in real time to update the test case execution order. The specific process is as follows Figure 2 As shown, including: 1. Obtain and analyze the test case set and historical execution logs.

[0089] (1) The product test case set is divided into text test cases + automated test cases; (2) Read the text test case prerequisites, test steps, expected results, and identify the dependencies between test cases; (3) Using the code2flow tool to identify the function code dependencies of automated test cases, and indirectly obtain the dependencies between test cases; (4) Combined with historical execution logs, identify implicit dependencies between use cases, including shared database tables and resource usage.

[0090] 2. Build a dependency graph.

[0091] (1) Test cases are nodes, and dependencies are directed edges (e.g., A→B means B depends on A); (2) Label the weight of the edge, including the degree of dependency (strong / weak), resource requirements (CPU / accelerator card), and execution time (the time required from the start of the test case to the completion of all steps of the test case); (3) Combine historical data to dynamically adjust dependency strength. For example, frequently failing dependencies can be marked as "strong." Historical data includes historical version execution logs, test case execution results, and corresponding blocking reasons, bug numbers, and test case execution times. In historical versions, if the test case execution result is PASS, it means the test case has passed. If the test case execution result is FAIL, it means the test case has failed. The corresponding bug number needs to be associated. If the test case execution result is BLOCK, it means the test case has been blocked. The reason for the blocked test needs to be noted.

[0092] The dependencies and dependency strengths between test cases can be obtained based on the test case execution results: In historical execution, when a test case fails, one or more other test cases often fail at the same time. This indicates that there is a dependency between this test case and the test cases that often fail at the same time, and the dependency can be adjusted. In historical execution, when a test case fails, it is associated with BUG number 123, and one or more other test cases are blocked at the same time, and the blocking reason is BUG123, then there is a strong dependency between the other blocked test cases and this test case, and the dependency can be adjusted. The test case displays the execution time of the test case, which can be used to optimize the execution sequence. The historical execution log records the resource requirements of each test case, including CPU and accelerator card, which are used to adjust the edge weights.

[0093] 4. Optimize execution order.

[0094] (1) After obtaining the dependency graph, the product test case set can be divided into test cases with dependencies and test cases without dependencies. These two categories of test cases are executed in parallel and assigned to different testers to execute together. (2) When use cases have dependency relationships, topological sorting can be performed based on the use case priority and execution time (consistent with the above execution time) so that use cases with high priority and short execution time are executed first; (3) When there is no dependency between use cases, group the non-dependent use cases (e.g. Group1: T1, T2; Group2: T3), and each group can be executed in parallel.

[0095] Test cases are sorted using the multi-objective optimization algorithm NSGA-II. The decision variable is the order in which the test cases are executed. Objective 1 (maximization) is the sum of the test case priorities (higher-priority cases are prioritized), and Objective 2 (minimization) is the total execution time (shorter cases are prioritized to reduce blocking). The constraint is that each case can only be executed once. A permutation encoding scheme is used, such as [3,1,2], which indicates that case 3 is executed first, followed by case 1, and finally case 2. The optimization factors are case priority and execution time. This maximizes resource utilization, allocating resources to each parallel group to prevent CPU, accelerator, and other resources from exceeding the environment's upper limit, which could lead to execution blocking. (4) Update the execution order based on the execution results of the real-time use case. For example, if the use case fails to execute or is blocked due to insufficient equipment, the use case downstream in the dependency chain will be skipped. (5) If it is detected that the execution time of a use case is far longer than expected, the priority of all subsequent dependent use cases (including the priority of directly dependent use cases and indirectly dependent use cases) will be dynamically reduced.

[0096] 5. Dependency loop detection.

[0097] Detect loops (such as A→B→C→A) and automatically decompose them into atomic tasks A, B, and C, which are then divided into three subtasks to reduce manual intervention.

[0098] Based on the above test case execution method, this application also constructs a test case execution system, the system architecture is as follows Figure 3 As shown, it includes dependency analyzer, graph generator, scheduling engine, and monitoring feedback module.

[0099] The dependency analyzer combines static text test cases and automated test case parsing with dynamic execution log analysis to improve the accuracy of dependency relationships. Graph Generator: Builds weighted dependency graphs, supports visual editing, and can manually correct incorrect dependencies. By adding manual calibration, the accuracy of the dependency graph can be improved. Scheduling engine: Generates execution order based on optimization algorithms and supports parallel and distributed execution. Parallel execution refers to multiple testers testing simultaneously, while distributed execution refers to multiple test environments, allowing testers to test in multiple test environments. Monitoring and feedback module: monitor execution results in real time and dynamically adjust subsequent sequences.

[0100] In summary, the test case execution method of the embodiment of the present application mainly includes: 1. Graph construction using a combination of dynamic and static methods: Combine text test cases, automated test cases, and historical execution logs to improve graph construction using a combination of dynamic and static methods; 2. Dynamically adjust the dependency graph: Dynamically adjust the dependency strength based on historical data (such as failure frequency and resource contention times); 3. Resource conflict prediction algorithm: This algorithm avoids parallel execution of high-resource-consuming use cases based on resource requirements. When building the dependency graph, it labels the edge weights and, based on historical data, labels the resource requirements of each use case. After completing the dependency graph, it allocates resources to each parallel group during task sequence optimization to prevent resources such as CPUs and accelerator cards from exceeding the environment's upper limits. 4. Feedback-driven real-time scheduling: Automatically update the execution order when execution fails.

[0101] 5. Circular dependency self-processing: Automatically detect circular dependencies and break them down into atomic tasks, reducing manual intervention The test case execution method of this application can achieve: 1. Efficiency improvement: Through dynamic dependency analysis and parallel optimization, test efficiency is improved, resource consumption is reduced, and test time is shortened; 2. Resource utilization optimization: Reduce resource conflicts and avoid execution interruptions caused by CPU / accelerator card contention; 3. Dynamically adjust the execution order: If the use case execution fails or is blocked, the execution order of the downstream dependency chain can be automatically adjusted; 4. Intelligently identify circular dependencies and automatically resolve them.

[0102] According to the test case execution method proposed in the embodiment of the present application, the explicit dependencies between test cases can be identified based on the test case set, the implicit dependencies between test cases can be identified based on the historical execution data, and the dependency graph of the test case set can be constructed based on the identification results of the explicit dependencies and implicit dependencies. The dependency graph is constructed by combining a static test case set with dynamic historical execution data, so that the constructed dependency graph is more accurate, and then the dependency type of the test case is determined based on the dependency graph, and the execution order of the test cases in the test case set is determined based on the dependency type, so as to take into account the different characteristics of test cases with different dependency types, improve the test efficiency, and then execute the test in the execution order to avoid executing invalid tests. To a certain extent, the test time can be shortened and the test cost can be reduced.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0104] An embodiment of the present application also provides a test case execution device.

[0105] Figure 4A schematic diagram of a test case execution device provided according to an embodiment of the present application.

[0106] like Figure 4 As shown, the test case execution device 10 includes: an acquisition module 100 , a first identification module 200 , a second identification module 300 , a construction module 400 and an execution module 500 .

[0107] Among them, the acquisition module 100 is used to obtain the historical execution data of the test case set and the test cases; the first identification module 200 is used to identify the explicit dependency relationship between the test cases based on the test case set; the second identification module 300 is used to identify the implicit dependency relationship between the test cases based on the historical execution data; the construction module 400 is used to construct a dependency graph of the test case set according to the identification results of the explicit dependency relationship and the implicit dependency relationship, wherein the dependency graph includes directed edges and nodes, the nodes represent the test cases, and the directed edges represent the existence of dependencies between the test cases; the execution module 500 is used to determine the dependency type of the test cases in the test case set according to the dependency graph, determine the execution order of the test cases in the test case set according to the dependency type, and execute the test cases in the test case set according to the execution order.

[0108] In some embodiments of the present application, the test case set includes text test cases and automated test cases.

[0109] In some embodiments of the present application, the first identification module 200 is further used to: read at least one of the preconditions, test steps, and expected results in a text test case, and determine the explicit dependency relationship between the text test cases based on at least one of the preconditions, test steps, and expected results; identify function code dependency data of automated test cases, and determine the explicit dependency relationship between automated test cases based on the function code dependency data.

[0110] In some embodiments of the present application, the historical execution data includes historical execution logs and historical data.

[0111] In some embodiments of the present application, the second identification module 300 is further used to: identify at least one of the shared database tables and resource occupancy of test cases in the historical execution log; and determine the implicit dependency between test cases based on at least one of the shared database tables and resource occupancy.

[0112] In some embodiments of the present application, the construction module 400 is further used to: determine the existence of dependencies between test cases based on the identification results of explicit dependencies and implicit dependencies; use test cases as nodes, and if there are dependencies between test cases, generate directed edges between the nodes corresponding to the test cases; if there are no dependencies between the test cases, generate nodes corresponding to the test cases.

[0113] In some embodiments of the present application, the test case execution device 10 of the embodiment of the present application further includes: an adjustment module.

[0114] Among them, the adjustment module is used to identify the execution results and problem data of the test cases in the historical data before determining the dependency types of the test cases in the test case set according to the dependency graph; and adjust the weights of the directed edges based on the execution results and problem data.

[0115] In some embodiments of the present application, the execution module 500 is further used to: obtain the priority of the test case; identify at least one characteristic data in the weight of the directed edge between the test cases, wherein the characteristic data includes the dependency degree of the test case, the resource requirement of the test case and the execution time of the test case; determine the execution order of the test cases in the test case set based on the dependency type, the priority of the test case and at least one characteristic data of the test case.

[0116] In some embodiments of the present application, the execution module 500 is further used to: if the dependency type of the test case is the first type, determine the execution order of the test case based on the priority of the test case, the dependency degree of the test case, and the execution time of the test case; if the dependency type of the test case is the second type, determine the execution order of the test case based on the priority of the test case and the execution time of the test case.

[0117] In some embodiments of the present application, the execution module 500 is further used to: execute the first type of test cases based on the execution order of the test cases with the first type of dependency type; execute the second type of test cases based on the execution order of the test cases with the second type of dependency type.

[0118] In some embodiments of the present application, the test case execution device 10 of the present application further includes: a determination module.

[0119] Among them, the determination module is used to obtain the execution result of the test case currently being executed after executing the test cases in the test case set in the execution order; if the execution result is execution failure or execution blocking, and the dependency type of the test case currently being executed is the first type, the downstream test case of the test case currently being executed is determined based on the dependency relationship, and the downstream test case of the test case currently being executed is not executed.

[0120] In some embodiments of the present application, the test case execution device 10 of the present application further includes: a reduction module.

[0121] Among them, the reduction module is used to identify the execution time of the test case after executing the test cases in the test case set in the execution order; if there is a target test case whose execution time exceeds the preset time, the downstream test cases related to the target test case are determined based on the dependency relationship, and the priority of the downstream test cases related to the target test case is reduced.

[0122] In some embodiments of the present application, the test case execution device 10 of the present application further includes: a splitting module.

[0123] Among them, the splitting module is used to split the multiple test cases in the directed loop into multiple sub-test cases before determining the execution order of the test cases in the test case set based on the dependency type, the priority of the test case and at least one characteristic data in the test case, if the dependency type of the test case is the first type and the test case forms a directed loop between the corresponding nodes in the dependency graph.

[0124] It should be noted that, for the description of the features in the embodiment corresponding to the test case execution device, reference can be made to the relevant description of the embodiment corresponding to the test case execution method, and no further details will be given here.

[0125] The embodiment of the present application also provides an electronic device, such as Figure 5 As shown, it includes a memory 501 and a processor 502, the memory 501 stores a computer program, and the processor 502 is configured to run the computer program to execute the steps in any of the above test case execution method embodiments.

[0126] An embodiment of the present application further provides a non-volatile computer-readable storage medium, in which a computer program is stored, wherein the non-volatile computer program is configured to execute the steps of any of the above-mentioned test case execution method embodiments when running.

[0127] In an exemplary embodiment, the non-volatile computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0128] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned test case execution method embodiments are implemented.

[0129] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0130] The above is a detailed introduction to a test case execution method, electronic device, storage medium, and program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core ideas of the present application. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, several improvements and modifications may be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A test case execution method, characterized in that: The following steps are involved: Get test case suites and historical test case execution data; identifying explicit dependencies between test cases based on the test case set; Identifying implicit dependencies between the test cases based on the historical execution data; Constructing a dependency graph of the test case set based on the identification results of the explicit dependency relationship and the implicit dependency relationship, wherein the dependency graph includes directed edges and nodes, the nodes represent the test cases, and the directed edges represent the existence of dependency relationships between the test cases; Dependency types of the test cases in the test case set are determined according to the dependency graph, an execution order of the test cases in the test case set is determined according to the dependency types, and the test cases in the test case set are executed according to the execution order.

2. The test case execution method according to claim 1, characterized in that: The test case set includes text test cases and automated test cases, and identifying explicit dependencies between test cases based on the test case set includes: Reading at least one of a precondition, a test step, and an expected result in the text test case, and determining an explicit dependency relationship between the text test cases based on at least one of the precondition, the test step, and the expected result; Identify function code dependency data of the automated test cases, and determine explicit dependency relationships between the automated test cases based on the function code dependency data.

3. The test case execution method according to claim 1, wherein: The historical execution data includes historical execution logs and historical data, and identifying the implicit dependency relationship between the test cases based on the historical execution data includes: Identify at least one of a shared database table and resource occupancy of the test case in the historical execution log; An implicit dependency relationship between the test cases is determined based on at least one of the shared database table and the resource occupancy.

4. The test case execution method according to claim 1, wherein: The step of constructing a dependency graph of the test case set according to the identification results of the explicit dependency relationship and the implicit dependency relationship includes: Determining, based on the identification results of the explicit dependency and the implicit dependency, that there is a dependency between the test cases; Taking the test cases as nodes, if the dependency relationship exists between the test cases, generating directed edges between the nodes corresponding to the test cases; If the dependency relationship does not exist between the test cases, a node corresponding to the test case is generated.

5. The test case execution method according to claim 3, characterized in that: Before determining the dependency types of the test cases in the test case set according to the dependency graph, the method further includes: Identifying test case execution results and problem data in the historical data; The weight of the directed edge is adjusted based on the execution result and the problem data.

6. The test case execution method according to claim 1, characterized in that: The determining, according to the dependency graph, the dependency type of the test case in the test case set includes: Identifying directed edges and nodes in the dependency graph; If the directed edge exists at a node in the dependency graph, determining that the dependency type of the test case corresponding to the node is the first type; If there is no directed edge for a node in the dependency graph, it is determined that the dependency type of the test case corresponding to the node is the second type.

7. The test case execution method according to claim 6, characterized in that: Determining the execution order of the test cases in the test case set according to the dependency type includes: Obtaining the priority of the test case; Identifying at least one characteristic data of the weights of the directed edges between the test cases, wherein the characteristic data includes the dependency degree of the test cases, the resource requirements of the test cases, and the execution time of the test cases; An execution order of the test cases in the test case set is determined based on the dependency type, the priority of the test case and at least one characteristic data of the test case.

8. The test case execution method according to claim 7, characterized in that: The determining the execution order of the test cases in the test case set based on the dependency type, the priority of the test case, and at least one of the characteristic data in the test case includes: If the dependency type of the test case is the first type, determining the execution order of the test case based on the priority of the test case, the dependency degree of the test case, and the execution time of the test case; If the dependency type of the test case is the second type, the execution order of the test case is determined based on the priority of the test case and the execution time of the test case.

9. The test case execution method according to claim 6, characterized in that: Executing the test cases in the test case set according to the execution order includes: Execute the test cases of the first type based on the execution order of the test cases of the first type of dependency type; The test cases of the second type are executed based on the execution order of the test cases of the second type of dependency.

10. The test case execution method according to claim 1, wherein: After executing the test cases in the test case set in the execution order, the method further includes: Get the execution result of the currently executing test case; If the execution result is execution failure or execution blocking, and the dependency type of the currently executing test case is the first type, the downstream test case of the currently executing test case is determined based on the dependency relationship, and the downstream test case of the currently executing test case is not executed.

11. The test case execution method according to claim 1, wherein: After executing the test cases in the test case set in the execution order, the method further includes: identifying an execution time of the test case; If there is a target test case whose execution time exceeds the preset time, downstream test cases related to the target test case are determined based on the dependency relationship, and the priorities of the downstream test cases related to the target test case are lowered.

12. The test case execution method according to claim 8, characterized in that: Before determining the execution order of the test cases in the test case set based on the dependency type, the priority of the test case and at least one of the characteristic data in the test case, the method further includes: If the dependency type of the test case is the first type, and the test case forms a directed loop between corresponding nodes in the dependency graph, multiple test cases in the directed loop are split into multiple sub-test cases.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the test case execution method according to any one of claims 1 to 12 when executing the computer program.

14. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the test case execution method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the test case execution method according to any one of claims 1 to 12 are implemented.

Images