Software testing method and device, equipment and medium

Abstract

The invention provides a software testing method and device, equipment and a medium. The method comprises the following steps: in response to a test task request received by test equipment, determining resources required by each test case contained in the test task request according to the test task request; respectively carrying the test cases and the resource information required by the test cases in execution requests corresponding to the test cases, and adding the execution requests corresponding to the test cases into a test case execution waiting queue of the test equipment; sequentially obtaining execution requests corresponding to the to-be-executed current test cases in the test case execution waiting queue, and determining whether current idle resources of the test equipment meet resources required by the current test cases or not; if yes, creating an execution thread to execute the current test case, and updating current idle resources of the test equipment; if not, waiting for other execution threads to end and releasing occupied resources; wherein the execution threads execute the test cases in parallel.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to a software testing method, device, equipment and medium. Background Art

[0002] In the process of software project development, in order to test software functions and performance and ensure software quality, test cases need to be executed frequently. If test cases are executed serially (i.e., one test case is executed before the next test case is executed), the test time will generally be longer. When the current idle resources of the test equipment are sufficient, the strategy of asynchronous concurrent execution of test cases (i.e., executing multiple test cases at the same time) is generally adopted to improve resource utilization, reduce the overall execution time of test cases, and thus improve test efficiency. Although this method is more efficient than serial scheduling, the use of a fixed concurrency number also has disadvantages: if the asynchronous concurrency number is too high, the resources required by the test case exceed the current idle resources, which will cause the test case execution to slow down, and even cause the test equipment to freeze, and a large number of test case execution failures; if the asynchronous concurrency number is too low, the resources required by the test case are less than the current idle resources, and the current idle resources of the test equipment are not fully utilized, resulting in low test efficiency. Summary of the invention

[0003] In view of this, the embodiments of the present application are committed to providing a software testing method, apparatus, device and medium to improve testing efficiency.

[0004] In a first aspect, a software testing method is provided, the method comprising: in response to a test device receiving a test task request, determining the resources required for each test case contained in the test task request according to the test task request; carrying each test case and its required resource information in an execution request corresponding to the test case, and adding the execution request corresponding to each test case to a test case waiting execution queue of the test device; sequentially obtaining the execution requests corresponding to the current test case to be executed in the test case waiting execution queue, and determining whether the current idle resources of the test device meet the resources required for the current test case; if so, creating an execution thread to execute the current test case, and updating the current idle resources of the test device; if not, waiting for other execution threads to end and release the occupied resources; wherein each execution thread executes the test case in parallel.

[0005] In a second aspect, a software testing device is provided, which is applied to a testing device, and the device includes: a first determination module, configured to determine the resources required for each test case contained in the test task request in response to the test device receiving a test task request; a request processing module, configured to carry each test case and its required resource information in the execution request corresponding to the test case, and add the execution request corresponding to each test case to the test case waiting execution queue of the testing device; a second determination module, configured to sequentially obtain the execution requests corresponding to the current test cases to be executed in the test case waiting execution queue, and determine whether the current idle resources of the testing device meet the resources required for the current test case; a software testing module, configured to create an execution thread to execute the current test case and update the current idle resources of the testing device if the requirements are met; if not, wait for other execution threads to end and release the occupied resources; wherein each execution thread executes the test case in parallel.

[0006] According to a third aspect, a computing device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, execute the method described in the first aspect above.

[0007] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0008] According to a fifth aspect, a program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to implement the method described in the first aspect.

[0009] After the test device receives the test task request, the present application carries each test case and the resources required by the test task request in the execution request corresponding to the test case to obtain the test case waiting execution queue of the test device. Next, after sequentially obtaining the execution requests corresponding to the current test cases to be executed from the test case waiting execution queue, if the current idle resources of the test device meet the resources required by the current test case, an execution thread is created to execute the current test case and the current idle resources of the test device are updated; if the current idle resources of the test device do not meet the resources required by the current test case, wait for other execution threads to end and release the occupied resources. By judging whether the current idle resources of the test device meet the resources required by the current test case, the test cases executed in parallel can be dynamically adjusted, which improves the test efficiency compared to the fixed number of concurrency used in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1It is a schematic diagram of the architecture of the application scenario of the software testing method provided in the embodiment of the present application;

[0011] Figure 2 is a flowchart of a software testing method provided by an embodiment of the present application;

[0012] Figure 3 It is a schematic diagram of a software testing method for a single test device;

[0013] Figure 4 It is a schematic diagram of a software testing method for a distributed testing system;

[0014] Figure 5 is a schematic diagram of a software testing device provided in an embodiment of the present application;

[0015] Figure 6 It is a schematic diagram of the structure of the test equipment provided in the embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] The software testing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the test device 104 through a communication network. The data storage system can store the data that the test device 104 needs to process. The data storage system can be integrated on the test device 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The test device 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0018] As another achievable manner, the above-mentioned test device 104 may also be implemented by a computer terminal with strong computing capability.

[0019] Figure 2 A flowchart of a software testing method provided in an embodiment of the present application, in which the scheme is applied to Figure 1 Test equipment in Figure 1 The test device 104 shown in the figure is used as an example for explanation. Figure 2 As shown in , the method may include the following steps:

[0020] Step 201: In response to a test device receiving a test task request, determining resources required for each test case included in the test task request according to the test task request.

[0021] Step 203: Each test case and the resource information required by it are respectively carried in the execution request corresponding to the test case, and the execution request corresponding to each test case is added to the test case waiting execution queue of the test device.

[0022] Step 205: sequentially obtain execution requests corresponding to the current test cases to be executed in the test case waiting execution queue, and determine whether the current idle resources of the test device meet the resources required by the current test case.

[0023] Step 207: If satisfied, create an execution thread to execute the current test case and update the current idle resources of the test device; if not satisfied, wait for other execution threads to end and release occupied resources; wherein each execution thread executes the test case in parallel.

[0024] It can be seen from the above process that after the test device receives the test task request, the present application carries each test case and the resources required by the test task request in the execution request corresponding to the test case to obtain the test case waiting execution queue of the test device. Next, after sequentially obtaining the execution requests corresponding to the current test case to be executed from the test case waiting execution queue, if the current idle resources of the test device meet the resources required by the current test case, an execution thread is created to execute the current test case and the current idle resources of the test device are updated; if the current idle resources of the test device do not meet the resources required by the current test case, wait for other execution threads to end and release the occupied resources. By judging whether the current idle resources of the test device meet the resources required by the current test case, the test cases executed in parallel can be dynamically adjusted, which improves the test efficiency compared to the fixed number of concurrency used in related technologies.

[0025] An embodiment of the present application relates to a software testing framework for dynamically managing resources, which includes test equipment resource monitoring, resource collection and storage required for test cases, and test case scheduling.

[0026] Among them, test equipment resource monitoring is to dynamically collect all resources that need to be controlled in the test equipment, such as the free RAM capacity of the test equipment, the number of CPU cores, etc.

[0027] The resource collection required by the test case refers to the collection of the resources occupied by the test equipment when each test case is executed normally, such as the RAM capacity occupied by each test case when it is executed normally. The resource storage required by the test case refers to the storage of the collected resources required by each test case in the database or file system (corresponding to Figure 3 ) in the database or file system module.

[0028] The above information is stored in a database or file system in the form of tables, such as a preset test case information table and a preset test task information table.

[0029] The preset test case information table includes relevant information of each test case, including but not limited to the index of the test case, resources required by the test case, running commands, etc.

[0030] The preset test task information table is used to record the relevant information of the test results of each test case in a certain test task, and the relevant information includes but is not limited to the index of the test task, the index of the test case, the test result, etc.

[0031] During the software testing process, when the relevant information of each test case changes, the preset test case information table can be dynamically updated; when the test result of a certain test task is obtained, the preset test task information table can be updated according to the test result.

[0032] The following describes in detail each step in the above process and the effects that can be further produced in conjunction with the embodiments.

[0033] First, the above step 201, namely "in response to the test device receiving the test task request, determining the resources required for each test case included in the test task request according to the test task request" is described in detail in conjunction with the embodiment.

[0034] In this embodiment, in the test equipment (eg Figure 1 When the test device 104 shown receives a test task request, it determines the resources required for each test case included in the test task request according to the test task request. Among them, a test case refers to a basic unit for testing a specific software. Its content includes test objectives, test environment, running commands, test source code files, and test description documents.

[0035] In one example, from a terminal (eg Figure 1 The terminal 102 shown in the figure can receive the test task request through the network and also receive the test task information, which may include the index of each test case; next, according to the index of each test case, the resources required for the test case with the corresponding index are queried in the preset test case information table.

[0036] It should be noted that a table can be used to store the index of each test case. For example, the test task information may include a test case index table. When obtaining the resources required for each test case, the resources required for the test case with the corresponding index are queried from the preset test case information table based on the index of each test case in the test case index table.

[0037] Correspondingly, in this example, the resources required for each test case included in the test task request are determined according to the test task request, including:

[0038] Determine the index of each test case included in the test task request according to the test task request;

[0039] According to the index of each test case, the resources required by the test case with the corresponding index are queried from the preset test case information table.

[0040] The resources required by a test case are generally set according to the maximum resources required to execute the test case (i.e., the maximum resources occupied by the test case when running). During the software testing process, the value of the resources required by the test case is a dynamically changing value. For example, at certain specific moments, the value will reach a peak value (i.e., the maximum resources).

[0041] In one example, the required resources include at least one of the following: the required number of CPU cores, the required number of GPU cores, the required RAM capacity, the required hard disk capacity, and the required network bandwidth.

[0042] It should also be noted that the resources required by each test case during execution are not limited to the several resources described above.

[0043] During software testing, you can decide whether to manage a resource based on the number of resources and the degree of relevance of the resource to the test case. If the number of a resource is far greater than the test case execution needs and it is certain that it will not be exhausted, or if the test case execution does not consume the resource, then the resource can be unmanaged. For example, if the hard disk capacity is large enough and far exceeds the test case execution needs, then the hard disk capacity can be unmanaged; for another example, if the test case is executed locally and does not rely on the network, then the network bandwidth can be unmanaged.

[0044] It should be noted that the current idle resources of the test equipment also include various types of resources. Each test case selects the corresponding test equipment according to whether all the resources required by it are met, so as to ensure that it can be executed normally on the test equipment.

[0045] The required number of CPU cores and the required RAM capacity are used as examples in describing the required resources below. For other required resources, please refer to this description and will not be repeated here.

[0046] In one example, obtaining resources for each test case may include:

[0047] The required number of CPU cores can be determined by analyzing whether there are multiple threads in the test case source code and determining the required number of CPU cores based on the number of threads.

[0048] The required RAM capacity can be obtained in the following ways: For example, in a Linux operating environment, the time tool can be used to obtain the RAM capacity required for each test case.

[0049] In the embodiment of the present application, during the software testing process, the resources required for each test case whose test results are successful are continuously monitored and recorded so as to be used as the resources required for the test case in the next test task, thereby improving the stability of the overall test.

[0050] In one example, the number of CPU cores can be adaptively adjusted when the test case execution times out or the number of threads changes due to a change in the test case source code.

[0051] In one example, the resources required for each test case are updated, including:

[0052] Taking RAM as an example, before executing a test case for the first time, set the required RAM capacity for the test case in the preset test case information table; after executing the test case and the test result of the test case is a successful test, use the time tool to collect the RAM capacity required for the process during execution, and write the collected RAM capacity into the preset test case information table to update its corresponding initial value.

[0053] During the software testing process, a larger RAM capacity and a larger number of CPU cores will be set as the initial values ​​to prioritize the normal progress of the software testing.

[0054] Correspondingly, in this example, after executing the test task multiple times, updating the resources required for each test case also includes:

[0055] For the same test case, in multiple historical test tasks (ie, test tasks executed before the current test task), when the test case is successfully executed multiple times, the resources required for the test case in the current test task are determined based on the resources required for each execution of the test case.

[0056] In one example, the maximum value among the three successful executions of the test case is used as the resources required for the test case in the current test task.

[0057] The embodiment of the present application can determine the method of updating the resources required for the test case according to the specific software testing project, which will not be repeated here.

[0058] The test device in the embodiment of the present application is any test device in a distributed test system.

[0059] After determining the resources required by each test case in the test case waiting execution queue according to the test task request, the method further includes:

[0060] If the current idle resources of the test device do not meet the resources required by all the test cases included in the test task request, some test cases are split out from the test cases included in the test task request, and the split test cases are sent to other test devices; wherein the current idle resources of each test device to which the test cases are assigned meet the resources required by the assigned test cases;

[0061] For the remaining test cases after splitting, continue to execute the steps of carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device.

[0062] In an embodiment of the present application, when a test device is insufficient to support the resources required for all test cases (i.e., the test cases waiting to be executed queue), some test cases are split out from all test cases and these test cases are distributed to other test devices in the distributed test system. For the split out test cases, the steps of respectively carrying each test case and its required resource information in the execution request corresponding to the test case and adding the execution request corresponding to each test case to the test case waiting to be executed queue of the test device are continued.

[0063] In one example, the test device is the first test device in the distributed test system; after determining the resources required by each test case in the test case waiting execution queue according to the test task request, the method further includes:

[0064] Starting from the first test device, the test case waiting execution queue is split into test cases whose resources satisfy the current idle resources of each test device in turn, and each test case that satisfies each test device and its required resource information are carried in the execution request corresponding to the test case, so as to add the execution request to the test case execution sub-queue corresponding to each test device;

[0065] Each test case in the queue is executed for each test case to perform the step of creating an execution thread to execute the current test case.

[0066] The embodiment of the present application distributes the test case waiting execution queue to multiple test devices for execution, specifically including:

[0067] exist Figure 4In the process, if the current idle resources of the first test device are sufficient to satisfy all the test cases to be executed, each test case and the resource information required by it are carried in the execution request corresponding to the test case and sent to the first test device. If the current idle resources of the first test device are not enough to satisfy the test case waiting execution queue, some test cases are split out from the test case waiting execution queue to ensure that these test cases can occupy the current idle resources of the first test device as much as possible, but cannot exceed the current idle resources of the first test device. Then, these test cases and the resource information required by them are carried in the execution request corresponding to the test case and sent to the first test device. Next, query the current idle resources of the second test device and send the test task according to the situation. And so on, until the test case waiting execution queue is fully distributed when it is distributed to the i-th test device, then the child thread is started to wait for the test task to be completed. Wherein, i is an integer greater than 1.

[0068] Continuing from the above, the current idle resources of each test device to which the test cases are assigned meet the resources required by the assigned test cases, which means that the current idle resources of the first test device meet the resources required by the test cases split out from it, and the current idle resources of the i-th test device meet the resources required by the test cases split out from it.

[0069] The above step 203, namely "carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device" is described in detail below in conjunction with the embodiments.

[0070] In an embodiment of the present application, each test case and the resources required by it are carried in an execution request corresponding to the test case, and the execution request corresponding to each test case is sequentially added to a queue of test cases waiting for execution.

[0071] In one example, before adding the execution request corresponding to each test case to the test case waiting execution queue, each test case and the resources required by it are packaged into a data structure, and the data structure is carried in the execution request corresponding to each test case.

[0072] For example, the index of each test case and its required number of CPU cores, required RAM capacity, and running command are packaged into a data structure, and the data structure is carried in the execution request corresponding to each test case. The running command of the test case is used to execute the test case.

[0073] The above step 205, namely "obtaining execution requests corresponding to the current test cases to be executed in the test case waiting execution queue in sequence, and determining whether the current idle resources of the test device meet the resources required by the current test case" is described in detail below in conjunction with the embodiments.

[0074] In the embodiment of the present application, the execution request corresponding to the current test case to be executed is obtained from the test case waiting execution queue in sequence, and it is determined whether the current idle resources of the test device meet the resources required by the current test case. Here, the current idle resources of the test device can be the latest idle resources of the test device (such as the new idle resources described below).

[0075] Here, the latest idle resources may refer to updating the current idle resources of the test device in step 205 in at least one of the following ways: in the process of polling the test case waiting execution queue, the current idle resources of the test device in step 205 are updated before each test case is executed; or, the current idle resources of the test device in step 205 are updated according to real-time idle resources; or, the occupied resources are released after the test case is executed to update the current idle resources of the test device in step 205.

[0076] The embodiment of the present application adopts a polling method to obtain the current test case to be executed from the test case waiting execution queue; for example, a current test case is taken out from the test case waiting execution queue each time at a fixed frequency, and then it is determined whether the current idle resources of the test device meet the resources required by the current test case.

[0077] The current test case to be executed is a test case that has not been executed. The current test case may be a test case selected from a test case waiting to be executed queue. The execution request corresponding to the test case is a specific operation instruction issued by a tester or an automated testing tool according to the detailed description of the test case during the software testing process. These execution requests are intended to verify whether the software's functions, performance, security, etc. meet the established requirements and standards.

[0078] It should be noted that step 205 may occur at any stage of the software testing process, such as the initial stage (ie, taking out the first test case from the test case waiting execution queue).

[0079] In the embodiment of the present application, each time a current test case is obtained from the test case waiting execution queue, the current idle resources of the test device can be updated by the following formula, specifically:

[0080] New number of CPU cores = old number of CPU cores - number of CPU cores required for the current test case;

[0081] New RAM capacity = old RAM capacity - RAM capacity required by the current test case.

[0082] Among them, the old idle resources (i.e., before obtaining the current test case from the test case waiting execution queue) include the old number of CPU cores and the old RAM capacity; the new idle resources (i.e., after obtaining the current test case from the test case waiting execution queue) include the new number of CPU cores and the new RAM capacity.

[0083] In the embodiment of the present application, after obtaining the current test case from the test case waiting execution queue, the new idle resource is used as the current idle resource of the test device in step 205 (corresponding to Figure 3 Modification (2) in the implementation process, a read-write lock is set on the current idle resources of the test device, and the query and modification of the current idle resources of the test device are bound together, that is, Figure 3 In the query (2) and modification (2), between the query (2) and modification (2), no other threads can query or modify the current idle resources of the test device.

[0084] During the software testing process, the current idle resources of the test device in step 205 are determined based on the recorded current idle resources.

[0085] The current idle resources recorded are determined using the following formula (1):

[0086] The recorded current free resources = the total amount of used resources data * the over-issuance coefficient - the resources occupied by the test case being executed.

[0087] During the software testing process, the resources required by each test case will not reach the maximum value (i.e., maximum resources) at the same time. Most of the time, the resources required by each test case during execution are less than the maximum resources, so the real-time idle resources are greater than the recorded current idle resources.

[0088] Based on the above reasons, the resources of the test equipment can be over-issued appropriately. For example, when the actual RAM of the test equipment is 16G, the test framework can be multiplied by an over-issue coefficient. For example, the over-issue coefficient is 120%. At this time, the over-issued RAM is 16G*120%=19.2G.

[0089] Here, "19.2G" can be used as the initial value of the recorded current idle resources. During the software testing process, the initial value can be updated based on formula (1) to update the recorded current idle resources.

[0090] In the embodiment of the present application, the over-issuance coefficient can be appropriately set according to the actual usage scenario. The over-issuance coefficient is greater than or equal to 1. However, the over-issuance coefficient should not be set too large to avoid test failure due to exhaustion of real resources.

[0091] During software testing, it is necessary to set a read-write lock on the recorded current idle resources to bind the query and modification of the recorded current idle resources together, that is, Figure 3 In the query (1) and modification (1), between the query (1) and modification (1), no other threads can query or modify the recorded current free resources.

[0092] During software testing, the following two abnormal scenarios may occur, which may lead to resource exhaustion: one is that one or some test cases occupy more than the required resources recorded during execution (corresponding to the resources occupied by the test cases being executed); the other is that other user processes not supervised by the test system occupy additional resources. Both types of abnormalities will cause the real-time free resources to be less than the recorded current free resources. If resources are still allocated according to the recorded current free resources, resource exhaustion will occur and the test case execution will fail. In order to avoid such scenarios, the following strategies are adopted:

[0093] The real-time idle resources of the test equipment are collected. If the collected real-time idle resources are less than the recorded current idle resources, the recorded current idle resources are updated using the product of the collected real-time idle resources and the over-issuance coefficient.

[0094] Here, the real-time idle resources of the test device may be actual real-time idle resources of the test device.

[0095] During the software testing process, the real-time idle resources of the test equipment are collected at a fixed frequency (for example, every 1 second). After each collection, they are compared with the recorded current idle resources. If the collected real-time idle resources are less than the recorded current idle resources, the recorded current idle resources are updated according to the real-time idle resources, so as to avoid over-issuance of resources of the test equipment and improve the fault tolerance and stability of the test equipment.

[0096] For example, when the real-time idle resources include 4G RAM, the test framework can multiply an over-issuance coefficient, such as 120%, and the over-issuance idle resources (i.e., the real-time idle resources after over-issuance) are 4G*120%=4.8G. 4.8G can be updated as the recorded current idle resources.

[0097] Optionally, after over-issuing the real-time idle resources, it is possible to confirm again whether the over-issued idle resources (i.e., the resources after multiplying the real-time idle resources by the over-issuance coefficient) are less than the recorded current idle resources. If the over-issued idle resources are less than the recorded current idle resources, the recorded current idle resources can be updated according to the over-issued idle resources.

[0098] The following is a detailed description of step 207 in conjunction with an embodiment, namely, "If satisfied, create an execution thread to execute the current test case and update the current idle resources of the test device; if not satisfied, wait for other execution threads to end and release occupied resources; wherein each execution thread executes the test case in parallel."

[0099] In this embodiment, when the current idle resources of the test device meet the resources required by the current test case, a thread (such as a child thread) is created for the current test case, and the current test case is placed in a thread pool to execute the running command of the current test case to execute the current test case, and after the current test case is executed, the current idle resources of the test device are updated; when the current idle resources of the test device do not meet the resources required by the current test case, wait for other execution threads to end and release the occupied resources. In the software testing process, the execution threads executed in parallel can be dynamically determined through the above operations.

[0100] Continuing from the above, the embodiment of the present application adopts a polling method to obtain the current test case to be executed from the test case waiting execution queue. The time for the software testing framework used in the embodiment of the present application to create an execution thread to execute the current test case is much longer than the time for polling the test case.

[0101] The embodiment of the present application adopts a polling method to obtain the current test case to be executed from the test case waiting execution queue, and creates a sub-thread of the current test case to execute the current test case each time the current idle resources of the test device meet the resources required by the current test case. Since the time for polling the test case is much shorter than the time for creating an execution thread to execute the current test case, multiple test cases have been polled in the process of the sub-thread executing a test case, so that other parallel sub-threads will execute their corresponding test cases.

[0102] In one example, when the current idle resources of the test device do not meet the resources required by the current test case and the current test case is executed, the resources required by the current test case can be re-added to the current idle resources of the test device by the following formula, so that the test case can wait for other test cases in the execution queue (i.e., other test cases to be executed) to apply for use:

[0103] New number of CPU cores = old number of CPU cores + number of CPU cores required for the current test case;

[0104] New RAM capacity = old RAM capacity + RAM capacity required by the current test case;

[0105] Among them, the old idle resources (i.e., the current idle resources before releasing the occupied resources) include the old number of CPU cores and the old RAM capacity; the new idle resources (i.e., the current idle resources after releasing the occupied resources) include the new number of CPU cores and the new RAM capacity.

[0106] After releasing the occupied resources, the new idle resources are used as the current idle resources of the test device in step 205 (corresponding to Figure 3 Modification (3) in the implementation process, a read-write lock is set on the current idle resources of the test device, and the query and modification of the current idle resources of the test device are bound together, that is, Figure 3 In the query (3) and modification (3), between the query (3) and modification (3), no other threads can query or modify the current idle resources of the test device.

[0107] In one example, when the current idle resources of the test device do not meet the resources required by the current test case, the method further includes: placing the current test case back to the end of the test case waiting execution queue, and continuing to obtain the next execution request from the test case waiting execution queue.

[0108] When the current idle resources of the test device do not meet the resources required by the current test case, the embodiment of the present application adopts the strategy of placing the current test case back to the end of the test case waiting execution queue to rearrange the execution order of the test cases, thereby ensuring that resources are used more efficiently while maintaining the continuity and integrity of the test.

[0109] Continuing from the above, the present application adopts a preset test case information table to store the resources required for each test case. After executing the current test case, the method further includes:

[0110] When the test result corresponding to the current test case is a test success, the resources occupied by the current test case during runtime and the index corresponding to the current test case are associated and written into the preset test case information table.

[0111] It should be noted that the resources required by the test case are used to represent the resources theoretically required for the normal execution of the test case during the test process; the resources occupied when the test case is running are used to represent the resources actually occupied during the test process.

[0112] exist Figure 3 In the example, when the test result is that the test is successful, the resources occupied by the current test case during execution and the index corresponding to the current test case are associated and written into the preset test case information table.

[0113] Continuing from the above, the present application adopts a preset test task information table to store the test results of each test case. After executing the current test case, the method further includes: when the test result corresponding to the current test case is a test success, the test result corresponding to the current test case and the index corresponding to the current test case are associated and written into the preset test task information table;

[0114] When the test result corresponding to the current test case is a test failure, the test result corresponding to the current test case and the index corresponding to the current test case are associated and written into the preset test task information table.

[0115] exist Figure 3 In the test, the test result corresponding to the current test case is written into the preset test task information table.

[0116] When the preset test task information table is queried to determine that all test cases included in the test task request have been executed, the embodiment of the present application generates a test report according to the test results corresponding to the indexes corresponding to the test cases in the preset test task information table.

[0117] In one example, in a test task waiting thread pool, each thread corresponds to a test task. The thread queries a preset test task information table in a database or file system at a fixed frequency to find out whether all test cases in the test task have been executed. After all test cases in the test task have been executed, a corresponding test report is generated according to the information in the preset test task information table in the database or file system, and sent to the designated location as needed.

[0118] Combination of the above Figure 2 , describes the method embodiment of the present application in detail, and the following is combined with Figure 5 , describes the device embodiment of the present application in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0119] Figure 5 FIG. 1 is a schematic diagram of the structure of a software testing device provided by an embodiment of the present application. Figure 5 As shown, the software testing device provided in the embodiment of the present application includes: a test task management module 501 and a test case execution module 502; wherein,

[0120] The test task management module 501 is configured to determine the resources required for each test case included in the test task request in response to the test device receiving the test task request;

[0121] The test case execution module 502 is configured to carry each test case and its required resource information in the execution request corresponding to the test case, and add the execution request corresponding to each test case to the test case waiting execution queue of the test device; obtain the execution request corresponding to the current test case to be executed in the test case waiting execution queue in turn, and determine whether the current idle resources of the test device meet the resources required by the current test case; if so, create an execution thread to execute the current test case, and update the current idle resources of the test device; if not, wait for other execution threads to end and release the occupied resources; wherein each execution thread executes the test case in parallel.

[0122] In some embodiments, the test case execution module 502 is also configured to put the current test case back to the end of the test case waiting execution queue if the current idle resources of the test device do not meet the resources required by the current test case, and continue to obtain the next execution request from the test case waiting execution queue.

[0123] In some embodiments, the test case execution module 502 is further configured to:

[0124] Collect the resources occupied by the test cases being executed;

[0125] The recorded current idle resources are determined based on the total resource usage data of the test device, the over-issuance coefficient, and the resources occupied by the test case being executed, where the over-issuance coefficient is greater than 1.

[0126] In some embodiments, the test case execution module 502 is further configured to:

[0127] Collect real-time idle resources of test equipment;

[0128] If the collected real-time idle resources are less than the recorded current idle resources, the recorded current idle resources are updated using the product of the collected real-time idle resources and the over-issuance coefficient.

[0129] In some embodiments, the test task management module 501 is further configured to:

[0130] According to the test task request, the index of each test case included in the test task request is determined, and the resources required by the test case corresponding to the index are queried from the preset test case information table stored in the database or file system module 503 .

[0131] In some embodiments, after executing the current test case, the system further includes:

[0132] The test case execution module 502 is also configured to write the resources occupied by the current test case during execution and the index association corresponding to the current test case into a preset test case information table stored in the database or file system module 503 if the test result corresponding to the current test case is a successful test.

[0133] In some embodiments, after executing the current test case, the system further includes:

[0134] The test case execution module 502 is further configured to write the test result corresponding to the current test case and the index corresponding to the current test case into a preset test task information table stored in the database or file system module 503 if the test result corresponding to the current test case is a test success;

[0135] If the test result corresponding to the current test case is a test failure, the test result corresponding to the current test case and the index corresponding to the current test case are associated and written into the preset test task information table stored in the database or file system module 503 .

[0136] In some embodiments, the system further comprises:

[0137] The test task management module 501 is also configured to query the preset test task information table to determine whether all test cases included in the test task request have been executed; when it is determined that all test cases have been executed, a test report is generated based on the test results corresponding to the indexes corresponding to each test case in the preset test task information table.

[0138] In some embodiments, the required resources include at least one of the following:

[0139] The required number of CPU cores, the required number of GPU cores, the required RAM capacity, the required hard disk capacity, and the required network bandwidth.

[0140] In some embodiments, the test device is any test device in a distributed test system.

[0141] In some embodiments, the test case execution module 502 is configured to split some test cases from the test cases included in the test task request if the current idle resources of the test device do not meet the resources required by all the test cases included in the test task request, and send the split test cases to other test devices; wherein the current idle resources of each test device to which the test case is assigned meet the resources required by the assigned test case;

[0142] For the remaining test cases after splitting, continue to execute the steps of carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device.

[0143] It should be noted that each test device in the distributed test system includes a test case execution module 502, which is used to provide an interface to facilitate the test task management module on the test device to query the current idle resources of the test device.

[0144] Below, reference Figure 6 To describe a computing device according to an embodiment of the present application. Figure 6 Shown is a schematic diagram of the structure of a computing device provided in one embodiment of the present application.

[0145] like Figure 6 As shown, the computing device includes one or more processors 601 and memory 602 .

[0146] The processor 601 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in a computing device to perform desired functions.

[0147] The memory 602 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 601 may run the program instructions to implement the functions of the software testing method of each embodiment of the present application mentioned above. Various contents such as model parameters and input data may also be stored in the computer-readable storage medium.

[0148] In one example, the computing device may further include: an input device 603 and an output device 604 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0149] The input device 603 may include, for example, a keyboard, a mouse, etc.

[0150] The output device 604 can output various information to the outside, including inference results, etc. The output device 604 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0151] Of course, to simplify, Figure 6 Only some of the components in the computing device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the computing device may further include any other appropriate components according to specific application scenarios.

[0152] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the software testing method according to various embodiments of the present application described above in this specification.

[0153] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the software testing method according to various embodiments of the present application described above in this specification.

[0155] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0157] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0158] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0159] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0160] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0161] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A software testing method, applied to a testing device, wherein: The method comprises: In response to the test device receiving a test task request, determining, according to the test task request, resources required for each test case included in the test task request; Carrying each test case and the resource information required by it in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device; Sequentially obtain execution requests corresponding to current test cases to be executed in the test case waiting execution queue, and determine whether current idle resources of the test device meet the resources required by the current test case; If the conditions are met, an execution thread is created to execute the current test case and the current idle resources of the test device are updated; if the conditions are not met, the execution thread waits for other execution threads to end and release the occupied resources; Among them, each execution thread executes the test cases in parallel.

2. The method according to claim 1, wherein: If the current idle resources of the test device do not meet the resources required by the current test case, the method further includes: placing the current test case back to the end of the test case waiting execution queue, and continuing to obtain the next execution request from the test case waiting execution queue.

3. The method according to claim 1 or 2, wherein: The current idle resources are determined in the following manner: The recorded current idle resources are obtained, where the recorded current idle resources are determined based on the total amount of resource usage data of the test device, an over-issuance coefficient, and resources occupied by the test case being executed, wherein the over-issuance coefficient is greater than 1.

4. The method according to claim 3, characterized in that The method further comprises: Collecting real-time idle resources of the test device; If the collected real-time idle resources are less than the recorded current idle resources, the recorded current idle resources are updated using the collected real-time idle resources and the over-issuance coefficient.

5. The method according to claim 1, wherein: The determining, according to the test task request, the resources required for each test case included in the test task request includes: Determine, according to the test task request, the index of each test case included in the test task request; According to the index of each test case, the resources required by the test case with the corresponding index are queried from the preset test case information table.

6. The method according to claim 5, wherein: After executing the current test case, the method further includes: If the test result corresponding to the current test case is a successful test, the resources occupied by the current test case and the index corresponding to the current test case are associated and written into the preset test case information table.

7. The method according to claim 1, wherein: After executing the current test case, the method further includes: The test result corresponding to the current test case and the index corresponding to the current test case are associated and written into the preset test task information table, wherein the test result includes a test success or a test failure.

8. The method according to claim 7, wherein: The method further comprises: Query the preset test task information table to determine whether all test cases included in the test task request have been executed; When it is determined that all the tests have been completed, a test report is generated according to the test results corresponding to the indexes corresponding to the test cases in the preset test task information table.

9. The method according to any one of claims 5 to 8, wherein: The required resources include at least one of the following: The required number of CPU cores, the required number of GPU cores, the required RAM capacity, the required hard disk capacity, and the required network bandwidth.

10. The method according to claim 1, wherein: The test device is any test device in a distributed test system.

11. The method according to claim 10, wherein: After determining the resources required by each test case in the test case waiting execution queue according to the test task request, the method further includes: If the current idle resources of the test device do not meet the resources required by all the test cases included in the test task request, some test cases are split out from the test cases included in the test task request, and the split test cases are sent to other test devices; wherein the current idle resources of each test device to which the test cases are assigned meet the resources required by the assigned test cases; For the remaining test cases after splitting, continue to execute the steps of carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device.

12. A software testing device, applied to a testing device, wherein: The device comprises: a test task management module and a test case execution module; wherein, The test task management module is configured to, in response to the test device receiving a test task request, determine the resources required for each test case included in the test task request according to the test task request; The test case execution module is configured to carry each test case and the resource information required by it in the execution request corresponding to the test case, and add the execution request corresponding to each test case to the test case waiting execution queue of the test device; Sequentially obtain execution requests corresponding to current test cases to be executed in the test case waiting execution queue, and determine whether current idle resources of the test device meet the resources required by the current test case; If the conditions are met, an execution thread is created to execute the current test case and the current idle resources of the test device are updated; if the conditions are not met, the execution thread waits for other execution threads to end and release the occupied resources; Among them, each execution thread executes the test cases in parallel.

13. A computing device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 11 by running the program in the memory.

14. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, cause the processor to implement the method according to any one of claims 1 to 11.

Images