Test case execution method and device, storage medium and program product

Abstract

The invention discloses a test case execution method and device, a storage medium and a program product, and relates to the technical field of computers.The test case execution method comprises the steps that resources needed by a first test case and first attribute information of all test steps in the first test case are used for indicating the resources needed by the first test case; and determining a first step which can be executed in combination with a second step in the main line test case in the first test case and a first execution priority of the first test case, so that when the first test case and the main line test case are executed in parallel, the similarity and compatibility among the test steps can be fully utilized; and the same test step is prevented from being repeatedly executed in a plurality of test cases. And based on the determined first execution priority and the merging identifier, the test sequence and the concurrency strategy can be dynamically adjusted. The technical problem of low test case execution efficiency in related technologies is solved, and the technical effect of improving the test case execution efficiency is achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method, device, storage medium, and program product for executing test cases. Background Art

[0002] Executing test cases is an important part of modern product quality assurance, which can help ensure that the performance, functionality, and reliability of software or hardware products meet the expected requirements. With the increasing complexity and integration of software and hardware systems, the number and complexity of test cases are also increasing continuously, making it difficult to arrange automated tests. Only automated scripts can be executed one by one, resulting in low overall test efficiency.

[0003] To improve test efficiency, it is usually necessary to optimize the execution process and strategy of test cases. However, most test case execution strategies rely on manual judgment, and the execution order of test cases cannot optimally utilize available resources. There are still technical problems with low efficiency in executing test cases. Summary of the Invention

[0004] This application provides a method, device, storage medium, and program product for executing test cases to at least solve the technical problem of low efficiency in executing test cases in related technologies.

[0005] This application provides a method for executing test cases, including: determining a first execution priority and a merge flag of a first test case based on first attribute information of the first test case included in a target test set, where the first test case is a test case allowed to be executed in parallel with the current main test case, the first attribute information is used to indicate resources required for executing the first test case and the content of each test step in the first test case, and the merge flag is used to indicate a second step included in the main test case allowed to be merged and executed with the first step in the first test case; when executing the first test case and the main test case in parallel, merging and executing the first step and the second step based on the first execution priority and the merge flag.

[0006] This application further provides an apparatus for executing test cases, including: a first determination module, configured to determine a first execution priority and a merge flag of a first test case based on first attribute information of the first test case included in a target test set, where the first test case is a test case allowed to be executed in parallel with the current main test case, the first attribute information is used to indicate resources required for executing the first test case and the content of each test step in the first test case, and the merge flag is used to indicate a second step included in the main test case allowed to be merged and executed with the first step in the first test case; a first execution module, configured to, when executing the first test case and the main test case in parallel, merge and execute the first step and the second step based on the first execution priority and the merge flag.

[0007] The present application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of the execution method of any one of the above test cases when executing the computer program.

[0008] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of the execution method of any one of the above test cases when executed by a processor.

[0009] The present application also provides a computer program product including a computer program, and the computer program implements the steps of the execution method of any one of the above test cases when executed by a processor.

[0010] Through the present application, by determining, according to the first attribute information indicating the resources required for the first test case and each test step in the first test case, the first step in the first test case that can be combined with the second step in the main test case for execution and the first execution priority of the first test case, it is possible to make full use of the similarity and compatibility between test steps when executing the first test case and the main test case in parallel, and avoid repeatedly executing the same test steps in multiple test cases. Moreover, based on the determined first execution priority and combination identifier, it is possible to dynamically adjust the test order and concurrency strategy. This solves the technical problem of low efficiency in executing test cases in the related art and achieves the technical effect of improving the execution efficiency of test cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] To illustrate the embodiments of the present application more clearly, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0012] Figure 1 It is a hardware structure block diagram of a mobile terminal for the execution method of a test case provided by an embodiment of the present application;

[0013] Figure 2 It is a flowchart of the execution method of a test case according to an embodiment of the present application;

[0014] Figure 3 It is a schematic diagram of the execution of the first test case and the main test case according to an embodiment of the present application;

[0015] Figure 4 It is a schematic diagram of a test set according to an embodiment of the present application;

[0016] Figure 5 It is a schematic diagram of the second test case according to an embodiment of the present application;

[0017] Figure 6 This is a flowchart of an execution method for a test case in a specific embodiment of the present application;

[0018] Figure 7 This is a schematic diagram of attribute information of a test case in a specific embodiment of the present application;

[0019] Figure 8 This is a schematic structural diagram of an execution device for a test case provided by an embodiment of the present application. Detailed implementation manners

[0020] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0021] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0022] In order to enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0023] Combined with a specific application environment architecture or a specific hardware architecture on which the execution of an execution method for a test case depends, the specific application environment architecture or the specific hardware architecture is described herein.

[0024] The method embodiments provided in the embodiments of the present application can be executed on a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for an execution method of a test case provided by an embodiment of the present application. As Figure 1 shown, the mobile terminal may include one or more ( Figure 1Only one processor 102 is shown (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a field programmable gate array FPGA), and a memory 104 for storing data. Among them, the above mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only schematic and does not limit the structure of the above mobile terminal. For example, the mobile terminal may further include more or fewer components than Figure 1 shown in, or have a different configuration from Figure 1 shown.

[0025] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to a test method for a data processing function in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely provided with respect to the processor 102, and these remote memories can be connected to the mobile terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one instance, the transmission device 106 includes a network interface controller (NIC for short), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF for short) module, which is used to communicate with the Internet wirelessly.

[0027] In this embodiment, a method for executing a test case is provided. Figure 2 It is a flowchart of a method for executing a test case in an embodiment of the present application. As Figure 2 shown, the process includes the following steps:

[0028] Step S202: Determine the first execution priority and merge identifier of the first test case based on the first attribute information of the first test case included in the target test set. Here, the first test case is a test case that is allowed to be executed in parallel with the current main test case. The first attribute information is used to indicate the resources required to execute the first test case and the content of each test step in the first test case. The merge identifier is used to indicate the second step included in the main test case that is allowed to be merged and executed with the first step in the first test case.

[0029] Optionally, the target test set refers to a set of test cases selected for automated testing of the device under test during a specific test cycle or test project. These test cases cover the key functions and performance indicators of the device under test, ensuring comprehensive and effective testing.

[0030] Optionally, the main test case is a test case that includes test steps that cannot be merged. Among them, it is possible to judge whether test steps can be merged according to the purpose of executing the test case, and it is also possible to directly judge whether test steps can be merged according to the attribute information of the test case. For example, test cases for stress testing of the Central Processing Unit (CPU for short), test cases for modifying different Basic Input / Output System (BIOS for short) selections, and test cases for testing the environment after modifying BIOS options are main test cases.

[0031] Optionally, the first test case is a test case that is allowed to be executed concurrently. Generally, it has low resource requirements or can share resources with the main test case without causing resource conflicts. For example, test cases for viewing Baseboard Management Controller (BMC for short) information, viewing Operating System (OS for short) information, and viewing information under stress to confirm whether certain functions are normal.

[0032] Optionally, the first execution priority is used to indicate the execution order level of executing the first test case. Generally, test cases with a higher execution priority will be given priority, especially when resources are limited.

[0033] Optionally, a merge identifier is a tag assigned to a specific test step in the first test case or the main test case, indicating that the first step can be merged with the second step for execution to save time and resources. For example, if test step 2 in the main test case can be merged with test step 2 in the first test case, then test step 2 in the main test case is the second step, and test step 2 in the first test case is the first step, and the merge identifier is the identifier that can identify the merged execution of test step 2 in the main test case and test step 2 in the first test case.

[0034] Optionally, the first attribute information can be determined from a database through an automated script.

[0035] Optionally, the resources required for the first test case include, but are not limited to, software resources, hardware resources, system status resources, and management interface resources required for executing the test case. For example, OS resources, BMC resources. The content of each test step in the first test case includes, but is not limited to, the identifier of the test step, the position of the test step in the first test case, and the expected result of the test step. For example, for a first test case of network card status check, the first attribute information may include: the presence of a network interface card hardware, the network connection of the OS, and test steps such as "send 100 ping requests" and "check whether the response time is within the threshold". The above first attribute information helps to determine the first execution priority of the first test case and the merge identifier of the first step that can be merged with the second step in the current main test case.

[0036] Step S204, when the first test case and the main test case are executed in parallel, based on the first execution priority and the merge identifier, merge and execute the first step and the second step.

[0037] Optionally, parallel execution means that during the testing of the device under test, multiple test cases (such as the first test case and the main test case) are executed simultaneously. Merge execution means combining two or more originally independent test steps into one execution process to reduce duplicate resource consumption and time overhead. As Figure 3 shown, Figure 3Execution schematic diagram of the first test case and the main test case of the embodiment of the present application. Among them, main test step 1 is test step 1 in the main test case, main test step 2 is test step 2 in the main test case, branch test step 1 is test step 1 in the first test case 1 that can be executed in combination with main test step 1, branch independent test step 2 is test step 2 in the first test case 1 that cannot be executed in combination with main test step 2, branch independent test step 1 is test step 1 in the first test case 2 that cannot be executed in combination with main test step 1, branch test step 2 is test step 2 in the first test case 2 that can be executed in combination with main test step 2. The merge identifier of the first test case 1 is 1, and the merge identifier of the first test case 2 is 2.

[0038] In this embodiment, the execution subject of the above steps may be a terminal, a server, a specific processor set in the terminal or the server, or a processor or processing device independently set relative to the terminal or the server, but is not limited thereto.

[0039] Through the embodiment of the present application, by using the first attribute information indicating the resources required for the first test case and each test step in the first test case, the first step that can be executed in combination with the second step in the main test case and the first execution priority of the first test case are determined. When the first test case and the main test case are executed in parallel, the similarity and compatibility between test steps can be fully utilized, and the repeated execution of the same test steps in multiple test cases can be avoided. Moreover, based on the determined first execution priority and merge identifier, the test order and concurrency strategy can be dynamically adjusted. The technical problem of low efficiency in executing test cases in the related art is solved, and the technical effect of improving the execution efficiency of test cases is achieved.

[0040] In an exemplary embodiment, before determining the first execution priority and merge identifier of the first test case based on the first attribute information of the first test case included in the target test set, the method further includes: determining multiple test cases for testing multiple devices to be tested, where the multiple test cases include the first test case and the main test case; performing a grouping operation on the multiple test cases to obtain multiple test sets, where different test sets correspond to different test environments; determining the target test set from the multiple test sets, where the multiple devices to be tested include a target device to be tested that can provide the target test environment, and the target test environment is the test environment required for each test case included in the target test set.

[0041] Optionally, the device under test is one or more devices that meet the preset device types. The devices under test include, but are not limited to, servers, storage devices, network devices, and security devices. The test cases include, but are not limited to, whole machine / component information check, stability test, stress test, performance test, function test, option modification, and firmware update. Specifically, all test cases for the device under test can be determined through the customer system interface, through database query, or through automated scripts. For example, through an automated script, the Basic Input / Output System (BIOS) tests for Server 1 and Server 2 are determined from the database to include information check, option modification, and stability test.

[0042] Optionally, when performing the grouping operation, all components required for the test case should be identified to determine whether there is a device under test that meets the test conditions for testing this test case. The test environment is used to indicate the software and hardware conditions set for executing the test case, including, but not limited to, the operating system status, hardware configuration, network connection status, software tools, and test data, etc.

[0043] Optionally, for example, the test cases for Device Under Test 1, Device Under Test 2, and Device Under Test 3 include: Test Case 1, Test Case 2, Test Case 3, Test Case 1a, Test Case 2a, Test Case 3a, Test Case 1b, Test Case 2b, and Test Case 3b. Among them, although Device Under Test 1 and Device Under Test 2 lack a Host Channel Adapter Card (hca card) and a Graphics Processing Unit Card (gpu card), and Device Under Test 3 lacks a hard disk backplane, they can provide the test environment for Test Case 1, Test Case 2, and Test Case 3; although Device Under Test 3 lacks a hard disk backplane, it can provide the test environment for Test Case 1a, Test Case 2a, and Test Case 3a; Device Under Test 1, Device Under Test 2, and Device Under Test 3 cannot provide the test environment for executing Test Case 1b, Test Case 2b, and Test Case 3b. Then, Test Case 1, Test Case 2, and Test Case 3 belong to Test Set a; Test Case 1a, Test Case 2a, and Test Case 3a belong to Test Set b; Test Case 1b, Test Case 2b, and Test Case 3b belong to Test Set c, as Figure 4 shown, Figure 4 is a schematic diagram of the test sets of the embodiments of the present application. The target test sets are Test Set a and Test Set b. The target devices under test for Test Set a are Device Under Test 1 and Device Under Test 2, and the target device under test for Test Set b is Device Under Test 3.

[0044] In this embodiment, by grouping and screening the test cases of multiple devices under test, multiple test sets are formed. Each test set targets a specific test environment, creating optimal test conditions for the test cases, achieving the purpose of avoiding the execution of test cases in unsuitable environments and reducing the environment configuration time before testing.

[0045] In an exemplary embodiment, after determining the target test set from multiple test sets and before parallelly executing the first test case and the main test case, the method further includes: determining a second test case from each test case included in the target test set based on the test type of each test case included in the target test set, where the second test case includes a first preset test step, and the execution priority of the first preset test step is higher than the execution priority of the test steps included in the first other test cases, and the first other test cases are the test cases included in the target test set except the second test case; executing the second test case.

[0046] Optionally, the test type is a type determined according to the purpose of the test case and the attribute information of the test case, where the attribute information includes but is not limited to the test steps and test environment of the test case. The test type includes but is not limited to a test type where some test steps cannot be merged and a test type where all test steps can be parallelized. The first preset test step is a test step used to provide time and environment for the execution of subsequent test cases. The test type of the second test case is a type that includes the first preset test step and where some test steps cannot be merged, and the execution priority of the second test case including the first preset test step is greater than the execution priority of the first other test cases that do not include the first preset test step. For example, a test case that includes modifying BIOS options.

[0047] Optionally, the first other test cases can be the first test case and the main test case.

[0048] Optionally, executing the second test case includes: merging the second test cases that include the same test steps to obtain a target second test case; executing the target second test case. As Figure 5 shown, Figure 5 is a schematic diagram of the second test case of the embodiment of the present application. The second test case 1, the second test case 2, and the second test case 3 are test cases that include modifying BIOS options. Among them, the above test cases include test steps of modifying the same BIOS options and restoring test options. The second test case 1, the second test case 2, and the second test case 3 can be merged into a merged case. It achieves the purpose of avoiding wasting time in separately modifying and restoring options for each test case and providing a test environment for the execution of other test cases.

[0049] In this embodiment, by preferentially executing a second test case including a key first preset test step, the test execution cycle can be effectively compressed. Especially when the first test case and the main test case are executed concurrently, the subsequent tests will not be hindered or delayed due to the incomplete basic test, achieving the purpose of effectively improving the test execution efficiency.

[0050] In an exemplary embodiment, before determining the first execution priority and the merge flag of the first test case, the method further includes: determining the main test case based on the test types of the respective test cases included in the target test set, where the main test case includes a second preset test step, the second preset test step is different from the first preset test step and the second preset test step includes steps that are not allowed to be executed in combination with the steps included in the second other test case, and the second other test case is a test case other than the main test case included in the target test set; determining the first execution priority and the merge flag of the first test case based on the first attribute information of the first test case included in the target test set includes: determining the test step association table between the first test case and the main test case based on the first attribute information and the second attribute information of the main test case, where the second attribute information is used to indicate the resources required to execute the main test case and the content of each test step in the main test case, and the test step association table includes the merge flag, the resources required to execute the third step other than the first step included in the first test case, and the resources required to execute the main test case; determining the first execution priority based on the test step association table, the first expected execution time of the first test case, and the second expected execution time of the main test case.

[0051] Optionally, the main test case is a test case including a second preset test step that cannot be executed in combination. Among them, it is possible to judge whether the test steps can be combined according to the purpose of executing the test case, and it is also possible to directly judge whether the test steps can be combined according to the attribute information of the test case. For example, the test cases for CPU stress testing, testing different BIOS selections, and testing the environment after modifying BIOS options are main test cases.

[0052] Optionally, the second attribute information can be determined from the database through an automated script.

[0053] Optionally, the second other test case can be the first test case and the second test case.

[0054] Optionally, the resources required for the main test case include, but are not limited to, software resources, hardware resources, system status resources, and management interface resources required to execute the test case. For example, OS resources and BMC resources. The content of each test step in the main test case includes, but is not limited to, the identification of the test step, the position of the test step in the main test case, and the expected result of the test step. The above first attribute information and second attribute information help determine the merge identification of the first step to be executed in combination with the second step and the resources required to execute the test step. Specifically, taking OS resources and BMC resources as examples, the resources required for the first test case and the main test case are described. The test step association table is shown in Table 1. The merge identification of the first test case 1 is 1, the merge identification of the first test case 2 is 2, and the merge identifications of the first test case 3 are 1 and 2.

[0055] Table 1:

[0056]

[0057]

[0058] Optionally, the first expected execution time can be obtained through the first attribute information, and the second expected execution time can be obtained through the second attribute information.

[0059] In this embodiment, the test step association table between the main test case and the first test case is determined through the test type, achieving the purpose of optimizing the test process according to the merge identification and resource requirements.

[0060] In an exemplary embodiment, based on the test step association table, the first expected execution time of the first test case, and the second expected execution time of the main test case, determining the first execution priority includes: based on the test step association table, determining the first step, the first resource required to execute the first step, the third step, and the second resource required to execute the third step; determining the first sub-time for executing the first step and the second sub-time for executing the third step based on the first expected execution time, and determining the first parameter based on the first sub-time, the second sub-time, and the second expected execution time; determining the second parameter based on the first resource, the second resource, and the third resource currently remaining in the target device under test; determining the first execution priority based on the first parameter and the second parameter.

[0061] Optionally, the method for determining the first step, the first resource, the third step, and the second resource based on the test step association table includes, but is not limited to: identification look-up method, condition filtering method, and data matching algorithm. For example, first determine the identification of the first test case, and then determine the merge identification of the first test case according to the identification of the first test case to screen out the first steps that can be executed in combination and the third steps that cannot be executed in combination. Then, look up the table to obtain the specific resources required for these steps, namely the first resource and the second resource.

[0062] Optionally, the first sub - time is used to indicate the test execution time of the first test case within the main test case time, and the second sub - time is used to indicate the test execution time of the first test case outside the main test case time. The methods for determining the first sub - time for executing the first step and the second sub - time for executing the third step based on the first expected execution time include, but are not limited to, the time - division - by - step method, the historical data statistics method, and the intelligent prediction algorithm.

[0063] Optionally, the first resource is used for the number of resources required for the test steps that can be executed in combination with the main test case, the second resource is used to indicate the number of resources required for the test steps that cannot be executed in combination with the main test case, and the third resource is used to indicate the maximum remaining concurrent resources of the target device under test.

[0064] Optionally, the methods for determining the first execution priority based on the first parameter and the second parameter include, but are not limited to, the weight - summation method, the weighted - average method, and the priority matrix method.

[0065] In this embodiment, the relationship between each test step and the required resources is clarified through the test - step association table, the resource requirements between the first test case and the main test case are analyzed, the first step and the third step and their corresponding resources are identified, and the execution of the first test case is effectively embedded into the process of the main test case, achieving the purpose of avoiding resource idleness and waste and improving resource utilization efficiency.

[0066] In an exemplary embodiment, determining the first parameter based on the first sub - time, the second sub - time, and the second expected execution time includes: determining the first parameter s through the following formula t :

[0067]

[0068] where t1 is the first sub - time, t2 is the second sub - time, and T is the second expected execution time.

[0069] Optionally, the first parameter can be negative, and the first parameter is used to indicate the execution priority of the first test case in terms of time. In this embodiment, the first parameter is determined through the above formula, accurately judging the test cases with less time consumption, and achieving the purpose of helping to reduce the test cycle and improve time utilization efficiency.

[0070] In an exemplary embodiment, determining the second parameter based on the first resource, the second resource, and the current remaining third resource of the target device under test includes: determining the second parameter s through the following formula c :

[0071]

[0072] Among them, C1 is the first resource, and C Max is the third resource, and C2 is the second resource.

[0073] Optionally, the second parameter is used to indicate the execution priority of the first test case in terms of resources. In this embodiment, the second parameter is determined by the above formula, accurately judging the test cases with less resource consumption, and achieving the purpose of helping to reduce the test cycle and improve the resource utilization efficiency.

[0074] In an exemplary embodiment, determining the first execution priority based on the first parameter and the second parameter includes: calculating the sum of the first parameter and the second parameter corresponding to each first test case respectively to determine the score of each first test case; sorting the scores of each first test case according to a preset sorting rule to obtain the sorting result of multiple first test cases. Among them, in the case where the scores of two first test cases are the same, the two first test cases are sorted according to the resource amount of the required predetermined resources they occupy; determining the priority of each first test case according to the sorting result.

[0075] Optionally, taking the determination of the priority based on the resources OS and BMC required for executing the first test case and the main line test case as an example, as shown in Table 2, the scores of the first test case a, the first test case b, the first test case c, and the first test case d are determined based on the sum of the first parameter and the second parameter, and the first test case a, the first test case b, the first test case c, and the first test case d are sorted to determine the priority of the above first test cases. If there are first test cases with the same score, they are sorted according to the OS concurrency score and the BMC concurrency. If they are still the same, they are sorted according to the first test case number.

[0076] Table 2:

[0077]

[0078] In an exemplary embodiment, after combining and executing the first step and the second step, the method further includes at least one of the following: in the case of determining that there are test cases that have been executed, deleting the executed test cases from the target test set; in the case of determining that the main line test case has been executed, re-determining the main line test case from the unexecuted test cases included in the target test set.

[0079] Optionally, in the case of determining that the main line test case has been executed, re-determining the main line test case from the unexecuted test cases included in the target test set includes: if all the main line test cases have been executed and there are remaining unexecuted first test cases, selecting the first test case with the lowest concurrency occupancy from the remaining unexecuted first test cases as the main line test case.

[0080] The present invention will be described below in conjunction with specific embodiments:

[0081] In this embodiment, taking the device to be tested as a server and the resources required for the first test case as OS resources and BMC resources as an example for explanation.

[0082] Figure 6 is a flowchart of an execution method of a test case in a specific embodiment of the present application. As Figure 6 shown, it includes the following steps:

[0083] S602. Since customer requirements are different, the server test configurations are different and the test cases are also different. Therefore, it is necessary to first determine all test cases. For example, for the restart test, the general requirement may be 200 rounds of restart. However, some customers require increasing the CPU network card pressure before restarting, and some customers require increasing the number of test times, such as 1000 rounds of testing or equivalent testing. All test cases required by customers for testing multiple servers of the same type or configuration are determined from the database through an automated script.

[0084] S604. Identify all components required for all the above test cases, perform a grouping operation on all test cases to obtain multiple test sets, so as to determine whether there is currently a server that meets the test conditions to test this test case. For example, if the configuration of the current server 1 meets the test conditions for executing test case A, test case B, test case C in test set 1 and test case D, test case E in test set 2, then server 1 is the target device to be tested, and test set 1 and test set 2 are the target test sets.

[0085] S606. According to the test steps and expected results in the attribute information of the test cases in the database, determine the second test cases that can be preferentially executed in test set 1 and test set 2. As Figure 7 shown. Among them, test case A is the second test case including modifying the BIOS option. Preferentially execute test case A to provide a test environment for executing other test cases. After test case A is executed, it is deleted from the target test set.

[0086] S608. As Figure 7 shown, according to whether the test steps in the attribute information of the test cases in the database are mergeable, determine the first test cases with all test steps being parallel and the main test cases including non-mergeable test steps. Among them, test case B and test case E are the main test cases, and test case C and test case D are the first test cases.

[0087] S610. Due to the concurrency limits of the server 1 OS and BMC, to improve the test execution efficiency, as many test cases C and D as possible need to be added within the concurrency numbers of the OS and BMC, and they should be executed in the way of combining the same test steps as much as possible. Specifically, first generate the test step association table of the main test cases without restart, and execute the main test cases without restart first. Secondly, generate the test step association table of the main test cases with restart, and then execute the main test cases with restart.

[0088] Among them, as Figure 7 shown, according to the test steps and expected results in the attribute information of the test cases in the database, it is determined that test case B is the main test case without restart, and test case E is the main test case with restart. Then, according to the OS status requirements, BMC status requirements, test steps and expected results in the attribute information, determine the test step association table of test case B. Among them, the test association table of test case B is shown in Table 3. The merge flag of test case C is 1, and the merge flag of test case D is 2:

[0089] Table 3:

[0090]

[0091] After determining the test step association table, input the OS and BMC required for executing the first step and the third step, as well as the remaining OS numbers and BMC numbers of server 1 into the following formula to calculate the first parameters of test cases C and D:

[0092]

[0093] Input the expected time required for executing the first step and the third step, as well as the expected time of test case D into the following formula to calculate the second parameters of test cases C and D:

[0094]

[0095] Based on the sum of the first parameter and the second parameter, determine the execution priorities of test cases C and D.

[0096] S612. For example, the execution priority of test case C is higher than that of test case D. During the test execution, check whether the remaining OS concurrency m of server 1 and the remaining BMC concurrency n can meet the concurrency requirements of test cases C and D. If it is satisfied, add test cases C and D to the execution process of test case B. The specific execution process is the same as Figure 3 the principle here and will not be elaborated. If only the concurrency requirement of test case D can be satisfied, then based on the merge flag of test case D being 2, merge and execute test cases D and B.

[0097] After test cases B and D are executed, they are deleted from the target test set. The target test set also includes the main test case: test case E and the first test case: test case C. Then, determine the test step association table of test case E to execute test cases C and E. If there are no test steps that can be combined and executed for test cases C and E, then when the remaining concurrency of server 1 meets the condition for concurrently executing test cases C and E, concurrently execute test cases C and E.

[0098] In this embodiment, by increasing concurrent testing as much as possible, combining the same test steps, accelerating the entire test cycle, reducing the number of steps repeatedly executed by the machine and the redundant concurrency, it realizes accelerating the delivery cycle of the server project and provides a faster and more efficient test solution for server manufacturers.

[0099] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0100] The embodiment of the present application also provides an execution device for test cases, as Figure 8 shown, including:

[0101] A first determination module 802, configured to determine the first execution priority and the merge flag of the first test case based on the first attribute information of the first test case included in the target test set, where the first test case is a test case that is allowed to be executed in parallel with the current main test case, the first attribute information is used to indicate the resources required to execute the first test case and the content of each test step in the first test case, and the merge flag is used to indicate the second step included in the main test case that is allowed to be combined and executed with the first step in the first test case;

[0102] A first execution module 804, configured to, when the first test case and the main test case are executed in parallel, combine and execute the first step and the second step based on the first execution priority and the merge flag.

[0103] In an exemplary embodiment, the above-mentioned device further includes: a second determination module, configured to determine multiple test cases for testing multiple devices under test before determining the first execution priority and the merge identifier of the first test case based on the first attribute information of the first test case included in the target test set, where the multiple test cases include the first test case and the main line test case; a second execution module, configured to perform a grouping operation on the multiple test cases to obtain multiple test sets, where different test sets correspond to different test environments; a third determination module, configured to determine a target test set from the multiple test sets, where the multiple devices under test include a target device under test that can provide the target test environment, and the target test environment is the test environment required for each test case included in the target test set.

[0104] In an exemplary embodiment, the above-mentioned device further includes: a fourth determination module, configured to determine a second test case from each test case included in the target test set based on the test type of each test case included in the target test set after determining the target test set from the multiple test sets and before parallelly executing the first test case and the main line test case, where the second test case includes a first preset test step, and the execution priority of the first preset test step is higher than the execution priority of the test steps included in the first other test cases, and the first other test cases are the test cases included in the target test set except the second test case; a third execution module, configured to execute the second test case.

[0105] In an exemplary embodiment, the above-mentioned device further includes: a fifth determination module, configured to determine the main line test case based on the test type of each test case included in the target test set before determining the first execution priority and the merge identifier of the first test case, where the main line test case includes a second preset test step, the second preset test step is different from the first preset test step and the second preset test step includes steps that are not allowed to be merged and executed with the steps included in the second other test cases, and the second other test cases are the test cases included in the target test set except the main line test case; the above-mentioned first determination module includes: a first determination sub-module, configured to determine a test step association table between the first test case and the main line test case based on the first attribute information and the second attribute information of the main line test case, where the second attribute information is used to indicate the resources required for executing the main line test case and the content of each test step in the main line test case, and the test step association table includes a merge identifier, the resources required for executing the third step except the first step included in the first test case, and the resources required for executing the main line test case; a second determination sub-module, configured to determine the first execution priority based on the test step association table, the first expected execution time of the first test case, and the second expected execution time of the main line test case.

[0106] In an exemplary embodiment, the second determination sub-module includes: a first determination unit, configured to determine a first step, a first resource required for executing the first step, a third step, and a second resource required for executing the third step based on a test step association table; a second determination unit, configured to determine a first sub-time for executing the first step and a second sub-time for executing the third step based on a first expected execution time, and determine a first parameter based on the first sub-time, the second sub-time, and a second expected execution time; a third determination unit, configured to determine a second parameter based on the first resource, the second resource, and a third resource remaining currently of a target device under test; and a fourth determination unit, configured to determine a first execution priority based on the first parameter and the second parameter.

[0107] In an exemplary embodiment, the second determination unit includes: a first determination sub-unit, configured to determine the first parameter s through the following formula t :

[0108]

[0109] where t1 is the first sub-time, t2 is the second sub-time, and T is the second expected execution time.

[0110] In an exemplary embodiment, the third determination unit includes: a second determination sub-unit, configured to determine the second parameter s through the following formula c :

[0111]

[0112] where C1 is the first resource, C Max is the third resource, and is the second resource.

[0113] In an exemplary embodiment, the fourth determination unit includes: a first calculation sub-unit, configured to calculate a sum value of the first parameter and the second parameter corresponding to each first test case respectively to determine a score of each first test case; a first sorting sub-unit, configured to sort the scores of each first test case according to a preset sorting rule to obtain a sorting result of multiple first test cases, where in a case where scores of two first test cases are the same, the two first test cases are sorted according to the resource amount of the required predetermined resources respectively; and a third determination sub-unit, configured to determine the priority of each first test case according to the sorting result.

[0114] In an exemplary embodiment, the above-mentioned device further includes at least one of the following: a first deletion module, configured to delete the executed test cases from the target test set when it is determined that there are executed test cases; a sixth determination module, configured to re-determine the main test cases from the unexecuted test cases included in the target test set when it is determined that the main test cases have been executed.

[0115] For the description of the features in the corresponding embodiment of an execution device for test cases, reference can be made to the relevant description in the corresponding embodiment of an execution method for test cases, which will not be elaborated here one by one.

[0116] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the execution method for test cases.

[0117] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above-mentioned embodiments of the execution method for test cases when running.

[0118] In an exemplary embodiment, the above-mentioned computer-readable storage medium may include, but is not limited to: various media that can store computer programs such as USB flash drives, read-only memories (ROM for short), random access memories (RAM for short), mobile hard disks, magnetic disks, or optical discs.

[0119] An embodiment of the present application further provides a computer program product. The above-mentioned computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-mentioned embodiments of the execution method for test cases.

[0120] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-mentioned embodiments of the execution method for test cases.

[0121] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0122] The above has introduced in detail a method for executing a test case provided by this application. Specific examples are used herein to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for executing test cases, characterized in that, Including: Based on the first attribute information of the first test case included in the target test set, determine the first execution priority and the merge identifier of the first test case, where the first test case is a test case that is allowed to be executed in parallel with the current main test case, the first attribute information is used to indicate the resources required to execute the first test case and the content of each test step in the first test case, and the merge identifier is used to indicate the second step included in the main test case that is allowed to be merged and executed with the first step in the first test case; When the first test case and the main test case are executed in parallel, based on the first execution priority and the merge identifier, merge and execute the first step and the second step.

2. The method according to claim 1, wherein Before determining the first execution priority and the merge identifier of the first test case based on the first attribute information of the first test case included in the target test set, the method further includes: Determine multiple test cases for testing multiple devices under test, where the multiple test cases include the first test case and the main test case; Perform a grouping operation on the multiple test cases to obtain multiple test sets, where different test sets correspond to different test environments; Determine the target test set from the multiple test sets, where the multiple devices under test include a target device under test that can provide a target test environment, and the target test environment is the test environment required for each test case included in the target test set.

3. The method according to claim 2, wherein After determining the target test set from the multiple test sets and before executing the first test case and the main test case in parallel, the method further includes: Based on the test types of the test cases included in the target test set, determine a second test case from the test cases included in the target test set, where the second test case includes a first preset test step, and the execution priority of the first preset test step is higher than the execution priority of the test steps included in the first other test cases, and the first other test cases are the test cases other than the second test case included in the target test set; Execute the second test case.

4. The method according to claim 3, wherein Before determining the first execution priority and the merge identifier of the first test case, the method further includes: based on the test types of the test cases included in the target test set, determine the main test case, where the main test case includes a second preset test step, the second preset test step is different from the first preset test step and the second preset test step includes steps that are not allowed to be merged and executed with the steps included in the second other test cases, and the second other test cases are the test cases other than the main test case included in the target test set; Determining the first execution priority and the merge identifier of the first test case based on the first attribute information of the first test case included in the target test set includes: determining a test step association table between the first test case and the main test case based on the first attribute information and the second attribute information of the main test case, where the second attribute information is used to indicate the resources required to execute the main test case and the content of each test step in the main test case, and the test step association table includes the merge identifier, the resources required to execute the third step other than the first step included in the first test case, and the resources required to execute the main test case; determining the first execution priority based on the test step association table, the first expected execution time of the first test case, and the second expected execution time of the main test case.

5. The method according to claim 4, wherein Determining the first execution priority based on the test step association table, the first expected execution time of the first test case, and the second expected execution time of the main test case includes: Based on the test step association table, determining the first step, the first resources required to execute the first step, the third step, and the second resources required to execute the third step; Determining a first sub-time for executing the first step and a second sub-time for executing the third step based on the first expected execution time, and determining a first parameter based on the first sub-time, the second sub-time, and the second expected execution time; Determining a second parameter based on the first resources, the second resources, and the third resources currently remaining in the target device under test; Determining the first execution priority based on the first parameter and the second parameter.

6. The method according to claim 5, wherein Determining a first parameter based on the first sub-time, the second sub-time, and the second expected execution time includes: The first parameter s is determined by the following formula t :[[]]END]] Where t1 is the first sub-time, t2 is the second sub-time, and T is the second expected execution time.

7. The method according to claim 5, characterized in that, Determining a second parameter based on the first resources, the second resources, and the third resources currently remaining in the target device under test includes: The second parameter s is determined by the following formula c :[[-END]] Among them, C1 is the first resource, and C Max is the third resource, and is the second resource.

8. The method according to claim 5, characterized in that, Determining the first execution priority based on the first parameter and the second parameter includes: Calculating the sum of the first parameter and the second parameter corresponding to each first test case respectively to determine the score of each first test case; Sorting the scores of each first test case according to a preset sorting rule to obtain the sorting results of the multiple first test cases, where, in the case where the scores of two first test cases are the same, sorting the two first test cases according to the amount of resources they occupy for the required predetermined resources; Determining the priority of each first test case according to the sorting result.

9. The method according to claim 1, characterized in that, After the first step and the second step are executed in combination, the method further includes at least one of the following: In the case where it is determined that there are test cases that have been executed, deleting the executed test cases from the target test set; When it is determined that the execution of the main-line test cases is completed, re-determine the main-line test cases from the unexecuted test cases included in the target test set.

10. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor for implementing the steps of the execution method of the test case according to any one of claims 1 to 9 when executing the computer program.

Images