Software testing method, apparatus, device, and medium

Abstract

This application provides a software testing method, apparatus, device, and medium. The method includes: responding to a test task request received by the test device; determining the resources required for each test case included in the test task request; carrying information about each test case and its required resources in the execution request corresponding to that test case, and adding the execution requests corresponding to each test case to the test device's test case waiting execution queue; sequentially retrieving the execution requests corresponding to the currently pending test cases in the test case waiting execution queue, and determining whether the test device's current idle resources meet the resources required for the current test case; if so, creating an execution thread to execute the current test case, and updating the test device's current idle resources; if not, waiting for other execution threads to finish and release the occupied resources; wherein, each execution thread executes the test cases in parallel.

Description

Technical Field

[0001] This application relates to the field of software testing technology, specifically to a software testing method, apparatus, equipment, and medium. Background Technology

[0002] In software project development, test cases need to be executed frequently to test software functionality and performance and ensure software quality. If test cases are executed sequentially (i.e., one test case is executed before the next), the testing time is generally quite long. When the test equipment has sufficient idle resources, an asynchronous concurrent execution strategy is typically adopted (i.e., executing multiple test cases simultaneously) to improve resource utilization, reduce the overall execution time of test cases, and thus improve testing efficiency. Although this method is more efficient than sequential scheduling, using a fixed concurrency level also has drawbacks: if the asynchronous concurrency level is too high, the resources required by the test cases exceed the currently available idle resources, leading to slower test case execution, or even causing the test equipment to freeze and a large number of test case failures; if the asynchronous concurrency level is too low, the resources required by the test cases are less than the currently available idle resources, resulting in insufficient utilization of the test equipment's currently available idle resources and low testing efficiency. Summary of the Invention

[0003] In view of this, the embodiments of this application aim to provide a software testing method, apparatus, device and medium to improve testing efficiency.

[0004] Firstly, a software testing method is provided, comprising: responding to a test device receiving a test task request, determining the resources required for each test case included in the test task request; carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution requests corresponding to each test case to the test case waiting execution queue of the test device; sequentially obtaining the execution requests corresponding to the current test cases to be executed in the test case waiting execution queue, and determining whether the current idle resources of the test device meet the resources required for the current test case; if they meet the requirements, creating an execution thread to execute the current test case, and updating the current idle resources of the test device; if they do not meet the requirements, waiting for other execution threads to finish and release the occupied resources; wherein, each execution thread executes the test cases in parallel.

[0005] Secondly, a software testing apparatus is provided, applied to a testing device. The apparatus includes: a first determining module configured to, in response to the testing device receiving a test task request, determine the resources required by each test case included in the test task request; a request processing module configured to carry each test case and its required resource information in the execution request corresponding to that test case, and add the execution requests corresponding to each test case to the test case waiting execution queue of the testing device; a second determining module configured to sequentially obtain the execution requests corresponding to the current test cases to be executed in the test case waiting execution queue, and determine whether the current idle resources of the testing device meet the resources required by the current test case; and a software testing module configured to, if the requirements are met, create an execution thread to execute the current test case and update the current idle resources of the testing device; if not, wait for other execution threads to finish and release the occupied resources; wherein each execution thread executes the test cases in parallel.

[0006] According to a third aspect, a computing device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, perform the method described in the first aspect above.

[0007] According to a fourth aspect, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect above.

[0008] According to a fifth aspect, a program product is provided, including computer program instructions that, when executed by a processor, cause the processor to implement the method described in the first aspect above.

[0009] This application, upon receiving a test task request, includes each test case and its required resources, contained within the test task request, in the execution request corresponding to that test case to obtain a test case execution queue for the test device. Next, after sequentially retrieving the execution requests corresponding to the current test case from the execution queue, if the test device's current idle resources meet the requirements of the current test case, an execution thread is created to execute the current test case, and the test device's current idle resources are updated. If the test device's current idle resources do not meet the requirements of the current test case, the application waits for other execution threads to finish and release their occupied resources. By determining whether the test device's current idle resources meet the requirements of the current test case, the number of test cases executed in parallel can be dynamically adjusted, improving testing efficiency compared to the fixed concurrency used in related technologies. Attached Figure Description

[0010] Figure 1This is a schematic diagram of the architecture of the software testing method provided in the embodiments of this application in an application scenario;

[0011] Figure 2 This is a flowchart of the software testing method provided in the embodiments of this application;

[0012] Figure 3 This is a schematic diagram of a software testing method for a single testing device;

[0013] Figure 4 This is a schematic diagram of software testing methods for a distributed testing system;

[0014] Figure 5 This is a schematic diagram of the software testing apparatus provided in the embodiments of this application;

[0015] Figure 6 This is a schematic diagram of the structure of the test equipment provided in the embodiments of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] The software testing method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with test equipment 104 via a communication network. A data storage system can store the data that test equipment 104 needs to process. The data storage system can be integrated into test equipment 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Test equipment 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0018] As another possible approach, the aforementioned test device 104 can also be implemented using a computer terminal with strong computing power.

[0019] Figure 2 A flowchart of the software testing method provided in the embodiments of this application, applied to the scheme. Figure 1 Test equipment in (e.g.) Figure 1 The following explanation uses the test device 104 as an example. Figure 2 As shown, the method may include the following steps:

[0020] Step 201: In response to the test device receiving the test task request, determine the resources required for each test case included in the test task request based on the test task request.

[0021] Step 203: Carry each test case and its required resource information into the execution request corresponding to the test case, and add the execution request corresponding to each test case to the test case waiting execution queue of the test device.

[0022] Step 205: Sequentially obtain the execution requests corresponding to the current test cases waiting to be executed in the test case waiting execution queue, and determine whether the current idle resources of the test device meet the resources required by the current test case.

[0023] Step 207: If satisfied, create an execution thread to execute the current test case and update the current idle resources of the test device; if not satisfied, wait for other execution threads to finish and release the occupied resources; wherein, each execution thread executes the test case in parallel.

[0024] As can be seen from the above process, after the test device receives a test task request, this application carries each test case and the resources required by it, contained in the test task request, in the execution request corresponding to that test case to obtain the test case waiting execution queue of the test device. Next, after sequentially retrieving the execution requests corresponding to the current test cases to be executed from the test case waiting execution queue, if the current idle resources of the test device meet the resources required by the current test case, an execution thread is created to execute the current test case, and the current idle resources of the test device are updated; if the current idle resources of the test device do not meet the resources required by the current test case, the process waits for other execution threads to finish and release the occupied resources. By determining whether the current idle resources of the test device meet the resources required by the current test case, the parallel execution of test cases can be dynamically adjusted, improving testing efficiency compared to the fixed concurrency used in related technologies.

[0025] This application relates to a software testing framework for dynamic resource management, which includes monitoring of test equipment resources, collection and storage of resources required for test cases, and test case scheduling.

[0026] Among them, test equipment resource monitoring involves dynamically collecting all resources within the test equipment that need to be managed, such as the idle RAM capacity and the number of CPU cores of the test equipment.

[0027] Resource acquisition for test cases refers to collecting the resources occupied by the test equipment during the normal execution of each test case, such as the RAM capacity used by each test case during normal execution. Resource storage for test cases refers to storing the collected resource requirements of each test case in a database or file system (corresponding to...). Figure 3 (Database or file system module in the system).

[0028] The above information can be stored in tables in a database or file system, such as a preset test case information table and a preset test task information table.

[0029] The preset test case information table includes relevant information for each test case, including but not limited to the test case index, the resources required for the test case, and the execution command.

[0030] A pre-defined test task information table is used to record relevant information about the test results of each test case in a test task. This relevant information includes, but is not limited to, the index of the test task, the index of the test case, and the test results.

[0031] During software testing, the preset test case information table can be dynamically updated when the relevant information of each test case changes; when the test result of a certain test task is obtained, the preset test task information table can be updated according to the test result.

[0032] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments.

[0033] First, the above step 201, namely "in response to the test device receiving a test task request, determine the resources required for each test case included in the test task request according to the test task request", will be described in detail with reference to the embodiments.

[0034] In this embodiment, in the test equipment (e.g. Figure 1 When the test device 104 shown receives a test task request, it determines the resources required for each test case included in the test task request. A test case is the basic unit for performing a test task on a specific piece of software. Its contents include the test objective, test environment, execution commands, test source code files, and test documentation.

[0035] In one example, from the terminal (e.g.) Figure 1 The terminal 102 shown can receive test task requests and test task information through the network. The test task information may include the index of each test case. Next, according to the index of each test case, the resources required by the test case corresponding to the index are queried in the preset test case information table.

[0036] It should be noted that a table can be used to store the index of each test case. For example, test task information may include a test case index table. When obtaining the resources required for each test case, the resources required for the test case with the corresponding index are queried from the preset test case information table for the index of each test case in the test case index table.

[0037] Correspondingly, in this example, the resources required for each test case included in the test task request are determined based on the test task request, including:

[0038] Determine the index of each test case included in the test task request based on the test task request;

[0039] Based on the index of each test case, the resources required by the corresponding test case are retrieved from the preset test case information table.

[0040] The resources required for a test case are generally set according to the maximum resources needed to execute that test case (i.e., the maximum resources occupied by the test case during runtime). However, during software testing, the resource requirements of a test case are a dynamically changing value; for example, this value may only reach its peak (i.e., the maximum resource) at certain specific times.

[0041] In one example, the required resources include at least one of the following: the required number of CPU cores, the required number of GPU cores, the required RAM capacity, the required hard disk capacity, and the required network bandwidth.

[0042] It should also be noted that the resources required by each test case during runtime are not limited to the resources described above.

[0043] During software testing, the decision to manage a resource can be based on its quantity and relevance to the test cases. If a resource's quantity far exceeds the needs of test case execution and is certain it won't be exhausted, or if test case execution won't consume the resource, then that resource can be left unmanaged. For example, if the hard drive capacity is large enough to far exceed the needs of test case execution, then the hard drive capacity can be left unmanaged; similarly, if test cases execute locally and don't depend on the network, then network bandwidth can be left unmanaged.

[0044] It should be noted that the currently available resources of the test equipment include various types of resources. Each test case selects the corresponding test equipment based on whether all of its required resources are met, in order to ensure that it can be executed normally on that test equipment.

[0045] The following description of required resources uses the number of CPU cores and the amount of RAM as examples. Other required resources can be referred to in this description and will not be repeated here.

[0046] In one example, retrieving resources for each test case may include:

[0047] The required number of CPU cores can be determined by analyzing whether there is multithreading in the test case source code and based on the number of threads.

[0048] The required RAM capacity can be obtained in the following ways: For example, in a Linux runtime environment, the time tool can be used to obtain the RAM capacity required for each test case.

[0049] In this embodiment of the application, during the software testing process, the resources required for each test case that is deemed a successful test are continuously monitored and recorded, so as to serve as the resources required for the test case in the next testing task, thereby improving the overall stability of the test.

[0050] In one example, the number of CPU cores can be adaptively adjusted when test case execution times out or when changes in the test case source code cause changes in the number of threads.

[0051] In one example, the resources required to update each test case include:

[0052] Taking RAM as an example, before executing a test case for the first time, the required RAM capacity is set in the preset test case information table for that test case; after the test case is executed and the test result is successful, the time tool is used to collect the RAM capacity required by the process during execution, and the collected RAM capacity is written into the preset test case information table to update its corresponding initial value.

[0053] During software testing, a large RAM capacity and a large number of CPU cores are set as initial values ​​to ensure the normal progress of software testing.

[0054] Correspondingly, in this example, updating the resources required for each test case after executing the test task multiple times also includes:

[0055] For the same test case, in multiple historical test tasks (i.e. test tasks executed before the current test task), when the test case is successfully executed multiple times, the resources required for the test case in the current test task are determined based on the resources required for each execution of the test case.

[0056] In one example, the maximum value among three successful executions of the test case is taken as the resource required for that test case in the current test task.

[0057] The embodiments of this application can determine the method of updating the resources required for test cases according to the specific software testing project, which will not be elaborated here.

[0058] In this embodiment of the application, the test device is any test device in a distributed test system.

[0059] After determining the resources required by each test case in the test case execution queue based on the test task request, the method further includes:

[0060] If the current available resources of the test equipment do not meet the resource requirements of all test cases included in the test task request, then a portion of the test cases are split from the test cases included in the test task request and sent to other test equipment; wherein the current available resources of each test equipment to which the test cases are assigned meet the resource requirements of the test cases to which they are assigned.

[0061] For the remaining test cases after splitting, continue to execute the steps of carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device.

[0062] In this embodiment of the application, when a test device is insufficient to support the resources required for all test cases (i.e., the test case waiting execution queue), a portion of the test cases will be split off from all the test cases and distributed to other test devices in the distributed test system. For the split test cases, the steps of carrying each test case and its required resource information in the execution request corresponding to the test case and adding the execution request corresponding to each test case to the test device's test case waiting execution queue will continue to be executed.

[0063] In one example, the test device is the first test device in the distributed testing system; after determining the resources required by each test case in the test case waiting execution queue based on the test task request, the method further includes:

[0064] Starting with the first test device, the test case waiting execution queue is split into test cases that meet the current idle resources of each test device. The test cases that meet the requirements of each test device and their required resource information are carried in the execution request corresponding to the test case, so that the execution request is added to the test case execution sub-queue corresponding to each test device.

[0065] For each test case in the test case execution queue, execute the step of creating an execution thread to execute the current test case.

[0066] This application embodiment distributes the test case execution queue to multiple test devices for execution, specifically including:

[0067] exist Figure 4In this process, if the current idle resources of the first test device are sufficient to satisfy all the test cases to be executed, then each test case and its required resource information are included in the execution request corresponding to that test case and sent to the first test device. If the current idle resources of the first test device are insufficient to satisfy the test case execution queue, then a portion of the test cases are split from the queue, ensuring that these test cases can occupy as much of the current idle resources of the first test device as possible, but not exceeding the current idle resources of the first test device. Then, these partial test cases and their required resource information are included in the execution request corresponding to that test case and sent to the first test device. Next, the current idle resources of the second test device are queried, and test tasks are dispatched according to the situation. This process continues until the i-th test device is reached, at which point the entire test case execution queue is completed, and a sub-thread is started to wait for the test tasks to complete. Here, i is an integer greater than 1.

[0068] Following the previous text, the current idle resources of each test device assigned test cases satisfying the resources required for the assigned test cases refer to: the current idle resources of the first test device satisfying the resources required for the test cases it splits out, and the current idle resources of the i-th test device satisfying the resources required for the test cases it splits out.

[0069] The following describes in detail step 203, namely, "carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device," with reference to the embodiments.

[0070] In this embodiment of the application, each test case and its required resources are carried in the execution request corresponding to the test case, and the execution requests corresponding to each test case are added to the test case waiting execution queue in sequence.

[0071] In one example, before adding the execution requests corresponding to each test case to the test case execution queue, each test case and its required resources are packaged into a data structure, and this data structure is carried in the execution requests corresponding to each test case.

[0072] For example, the index of each test case, along with its required number of CPU cores, required RAM capacity, and execution commands, can be packaged into a data structure, and this data structure can be carried in the execution request corresponding to each test case. The execution commands for each test case are used to execute the test cases.

[0073] The following describes in detail step 205, namely, "sequentially obtaining the execution requests corresponding to the current test cases waiting to be executed in the test case waiting execution queue, and determining whether the current idle resources of the test device meet the resources required by the current test case", with reference to the embodiments.

[0074] In this embodiment, the execution request corresponding to the current test case to be executed is sequentially retrieved from the test case waiting execution queue, and it is determined whether the current idle resources of the test device meet the resource requirements of the current test case. Here, the current idle resources of the test device can be the latest idle resources of the test device (as described below, new idle resources).

[0075] Here, the latest free resources can refer to updating the current free resources of the test device in step 205 in at least one of the following ways: during the polling of the test case waiting execution queue, the current free resources of the test device in step 205 are updated before each test case is executed; or, the current free resources of the test device in step 205 are updated according to the real-time free resources; or, the current free resources of the test device in step 205 are updated by releasing the occupied resources after the test case is executed.

[0076] In this embodiment, a polling method is used to obtain the current test case to be executed from the test case waiting execution queue; for example, a current test case is taken from the test case waiting execution queue at a fixed frequency each time, and then it is determined whether the current idle resources of the test device meet the resources required by the current test case.

[0077] The aforementioned "current test cases awaiting execution" are test cases that have not yet been executed. These current test cases can be selected from the test case execution queue. The execution requests corresponding to these test cases are specific operational instructions issued by testers or automated testing tools during software testing, based on the detailed description of the test cases. These execution requests aim to verify whether the software's functionality, performance, security, and other aspects meet predetermined requirements and standards.

[0078] It should be noted that step 205 can occur at any stage of the software testing process, such as the initial stage (i.e., retrieving the first test case from the test case waiting execution queue).

[0079] In this embodiment of the application, when a current test case is retrieved from the test case waiting execution queue, the current idle resources of the test device can be updated using the following formula:

[0080] New CPU core count = Old CPU core count - CPU core count required for the current test case;

[0081] New RAM capacity = Old RAM capacity - RAM capacity required for the current test case.

[0082] The old idle resources (i.e., before the current test case is retrieved from the test case waiting execution queue) include the old number of CPU cores and the old RAM capacity; the new idle resources (i.e., after the current test case is retrieved from the test case waiting execution queue) include the new number of CPU cores and the new RAM capacity.

[0083] In this embodiment of the application, after obtaining the current test case from the test case waiting execution queue, the new idle resource is used as the current idle resource of the test device in step 205 (corresponding to...). Figure 3 In the modification (2), during the implementation process, a read-write lock is applied to the currently idle resources of the test device, binding the query and modification of the currently idle resources of the test device together, that is... Figure 3 In the query (2) and modification (2) process, no other thread may query or modify the current idle resources of the test device between the query (2) and modification (2) process.

[0084] During software testing, the current idle resources of the test device in step 205 are determined based on the recorded current idle resources.

[0085] The currently available free resources are determined using the following formula (1):

[0086] Recorded current idle resources = total resources used * over-issuance coefficient - resources used by currently executing test cases.

[0087] During software testing, the resources required by each test case will not reach their maximum value (i.e., maximum resource) at the same time. Most of the time, the resources required by each test case during execution are less than the maximum resource, so the real-time idle resources are greater than the recorded current idle resources.

[0088] Based on the above reasons, the resources of the test equipment can be over-allocated appropriately. For example, when the actual RAM of the test equipment is 16G, the test framework can be multiplied by an over-allocation factor, such as an over-allocation factor of 120%. In this case, the over-allocated RAM is 16G * 120% = 19.2G.

[0089] Here, “19.2G” can be used as the initial value of the recorded current idle resources. During the software testing process, the initial value can be updated based on formula (1) to update the recorded current idle resources.

[0090] The embodiments of this application can appropriately set the over-issuance coefficient according to the actual use scenario. The over-issuance coefficient is greater than or equal to 1, but the over-issuance coefficient should not be set too large to avoid test failure due to the exhaustion of real resources.

[0091] During software testing, it is necessary to apply read-write locks to the currently available resources, binding the querying and modification of these resources together. Figure 3 In the query (1) and modification (1) process, no other thread can query or modify the currently available resources of the record between the query (1) and modification (1) process.

[0092] During software testing, two abnormal scenarios may occur that lead to resource exhaustion: First, one or more test cases consume more resources than recorded (corresponding to the resources currently being executed). Second, other user processes not monitored by the test system consume additional resources. Both of these anomalies result in real-time idle resources falling below the recorded current idle resources. If resource allocation is still based on the recorded current idle resources, resource exhaustion will occur, leading to test case execution failure. To avoid such scenarios, the following strategy is adopted:

[0093] The system collects real-time idle resources of the test equipment. If the collected real-time idle resources are less than the recorded current idle resources, the system updates the recorded current idle resources by multiplying the collected real-time idle resources by the over-issuance coefficient.

[0094] Here, the real-time idle resources of the test equipment can be the actual idle resources of the test equipment in real time.

[0095] During software testing, real-time idle resources of the test equipment are collected at a fixed frequency (e.g., every 1 second). After each collection, the data is compared with the recorded current idle resources. If the collected real-time idle resources are less than the recorded current idle resources, the recorded current idle resources are updated according to the real-time idle resources. This avoids over-exploitation of test equipment resources and improves the fault tolerance and stability of the test equipment.

[0096] For example, if the real-time idle resources, including RAM, are 4GB, the test framework can multiply this by an over-issuance factor, such as 120%. In this case, the over-issued idle resources (i.e., the real-time idle resources after over-issuance) will be 4GB * 120% = 4.8GB. This 4.8GB can then be updated to reflect the currently recorded idle resources.

[0097] Optionally, after over-issuing real-time idle resources, it can be confirmed again whether the over-issued idle resources (i.e., the resources after multiplying the real-time idle resources by the over-issuance coefficient) are less than the recorded current idle resources. If the over-issued idle resources are less than the recorded current idle resources, the recorded current idle resources can be updated according to the over-issued idle resources.

[0098] The following describes step 207, namely, "If satisfied, an execution thread is created to execute the current test case and the current idle resources of the test device are updated; if not satisfied, other execution threads are waited for to finish and the occupied resources are released; wherein, each execution thread executes the test case in parallel," in detail with reference to the embodiments.

[0099] In this embodiment, when the current idle resources of the test equipment meet the resource requirements of the current test case, a thread (such as a child thread) is created for the current test case, and the current test case is placed in a thread pool to execute the run command of the current test case. After the current test case is executed, the current idle resources of the test equipment are updated. When the current idle resources of the test equipment do not meet the resource requirements of the current test case, the system waits for other execution threads to finish and release the occupied resources. During software testing, the execution threads that execute in parallel can be dynamically determined through the above operations.

[0100] Following the previous text, this embodiment uses a polling method to retrieve currently pending test cases from the test case execution queue. The time required for the software testing framework used in this embodiment to create an execution thread to execute the current test case is significantly longer than the time required to poll for test cases.

[0101] In this embodiment, a polling method is used to obtain the current test case to be executed from the test case waiting execution queue. Each time the current idle resources of the test device meet the resources required by the current test case, a child thread for the current test case is created to execute the current test case. Since the time for polling test cases is much shorter than the time for creating an execution thread to execute the current test case, multiple test cases have already been polled while the child thread is executing a test case. This allows other child threads to execute their corresponding test cases in parallel.

[0102] In one example, if the current available resources of the test equipment do not meet the resources required by the current test case, and the current test case has finished executing, the resources required by the current test case can be added back to the current available resources of the test equipment using the following formula, so that the test case can wait for other test cases in the execution queue (i.e., other test cases waiting to be executed) to request its use:

[0103] New CPU core count = old CPU core count + CPU core count required for the current test case;

[0104] New RAM capacity = Old RAM capacity + RAM capacity required for the current test case;

[0105] The old free resources (i.e., the current free resources before the occupied resources are released) include the old number of CPU cores and the old RAM capacity; the new free resources (i.e., the current free resources after the occupied resources are released) include the new number of CPU cores and the new RAM capacity.

[0106] After releasing the occupied resources, the new free resources are used as the current free resources of the test device in step 205 (corresponding to...). Figure 3 In the modification (3), during the implementation process, a read-write lock is applied to the currently idle resources of the test device, binding the query and modification of the currently idle resources of the test device together, that is... Figure 3 In the query (3) and modification (3) process, no other thread may query or modify the current idle resources of the test device between the query (3) and modification (3) process.

[0107] In one example, if the current idle resources of the test device do not meet the resources required by the current test case, the method further includes: putting the current test case back to the tail of the test case waiting execution queue, and continuing to obtain the next execution request from the test case waiting execution queue.

[0108] In this embodiment of the application, when the current idle resources of the test equipment do not meet the resources required by the current test case, the current test case is placed back to the tail of the test case waiting execution queue to rearrange the execution order of the test cases. This can ensure that resources are used more effectively while maintaining the continuity and integrity of the test.

[0109] Following the above, this application uses a preset test case information table to store the resources required by each test case. After executing the current test case, the method further includes:

[0110] When the test result corresponding to the current test case is successful, the resources occupied by the current test case during runtime and the index corresponding to the current test case are written into the preset test case information table.

[0111] It should be noted that the resources required by a test case represent the resources theoretically required for the test case to execute normally during the test; the resources occupied during the runtime of a test case represent the resources actually occupied during the test.

[0112] exist Figure 3 In the test, when the test result is successful, the resources occupied by the current test case and the index corresponding to the current test case are written into the preset test case information table.

[0113] Following the above, this application uses a preset test task information table to store the test results of each test case. After the current test case is executed, the method further includes: when the test result corresponding to the current test case is successful, writing the test result corresponding to the current test case and the index corresponding to the current test case into the preset test task information table.

[0114] When the test result corresponding to the current test case is a test failure, the test result corresponding to the current test case and the index corresponding to the current test case are written into the preset test task information table.

[0115] exist Figure 3 In this process, the test results corresponding to the current test case are written into the preset test task information table.

[0116] In this embodiment of the application, when the preset test task information table is queried and it is determined that all test cases included in the test task request have been executed, a test report is generated based on the test results corresponding to the index of each test case in the preset test task information table.

[0117] In one example, in the test task waiting thread pool, each thread corresponds to one test task. The thread queries the database or file system's preset test task information table at a fixed frequency to find out whether all the test cases in the test task have been executed. After all the test cases in the test task have been executed, the corresponding test report is generated based on the information in the database or file system's preset test task information table and sent to the specified location as needed.

[0118] The above text combined Figure 2 The method embodiments of this application are described in detail below, in conjunction with... Figure 5 The present application provides a detailed description of the apparatus embodiments. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.

[0119] Figure 5 The diagram shown is a structural schematic of a software testing apparatus provided in an embodiment of this application. Figure 5 As shown, the software testing apparatus provided in this embodiment includes: a test task management module 501 and a test case execution module 502; wherein,

[0120] The test task management module 501 is configured to, in response to the test device receiving a test task request, determine the resources required for each test case included in the test task request based on the test task request.

[0121] The test case execution module 502 is configured to carry each test case and its required resource information in the execution request corresponding to the test case, and add the execution requests corresponding to each test case to the test case waiting execution queue of the test device; sequentially obtain the execution requests corresponding to the current test cases waiting to be executed in the test case waiting execution queue, and determine whether the current idle resources of the test device meet the resources required by the current test case; if they do, create an execution thread to execute the current test case, and update the current idle resources of the test device; if they do not meet, wait for other execution threads to finish and release the occupied resources; wherein, each execution thread executes the test cases in parallel.

[0122] In some embodiments, the test case execution module 502 is further configured to, if the current idle resources of the test device do not meet the resources required by the current test case, put the current test case back to the tail of the test case waiting execution queue and continue to obtain the next execution request from the test case waiting execution queue.

[0123] In some embodiments, the test case execution module 502 is further configured to:

[0124] Collect the resources used by the currently executing test cases;

[0125] The recorded current idle resources are determined based on the total resource usage data of the test equipment, the over-provisioning coefficient, and the resources occupied by the currently executing test cases, where the over-provisioning coefficient is greater than 1.

[0126] In some embodiments, the test case execution module 502 is further configured to:

[0127] Collect real-time idle resources of the test equipment;

[0128] If the collected real-time idle resources are less than the recorded current idle resources, then the recorded current idle resources are updated using the product of the collected real-time idle resources and the over-issuance coefficient.

[0129] In some embodiments, the test task management module 501 is further configured to:

[0130] Based on the test task request, determine the index of each test case included in the test task request, and query the preset test case information table stored in the database or file system module 503 to find the resources required by the test case corresponding to the index.

[0131] In some embodiments, after the current test case has been executed, the system further includes:

[0132] The test case execution module 502 is also configured to write the resources occupied by the current test case during runtime and the index corresponding to the current test case into a preset test case information table stored in the database or file system module 503 if the test result corresponding to the current test case is successful.

[0133] In some embodiments, after the current test case has been executed, the system further includes:

[0134] The test case execution module 502 is also configured to write the test result corresponding to the current test case and the index corresponding to the current test case into a preset test task information table stored in the database or file system module 503 if the test result corresponding to the current test case is successful.

[0135] If the test result corresponding to the current test case is a test failure, then the test result corresponding to the current test case and the index corresponding to the current test case are associated and written into the preset test task information table stored in the database or file system module 503.

[0136] In some embodiments, the system further includes:

[0137] The test task management module 501 is also configured to query a preset test task information table to determine whether all test cases included in the test task request have been executed; when it is determined that all test cases have been executed, a test report is generated based on the test results corresponding to the index of each test case in the preset test task information table.

[0138] In some embodiments, the required resources include at least one of the following:

[0139] Required number of CPU cores, required number of GPU cores, required RAM capacity, required hard disk capacity, and required network bandwidth.

[0140] In some embodiments, the test device is any test device in a distributed test system.

[0141] In some embodiments, the test case execution module 502 is configured to, if the current idle resources of the test device do not meet the resources required by all the test cases included in the test task request, split off a portion of the test cases from the test cases included in the test task request and send the split-off portion of the test cases to other test devices; wherein the current idle resources of each test device to which the test cases are assigned meet the resources required by the test cases to which they are assigned.

[0142] For the remaining test cases after splitting, continue to execute the steps of carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device.

[0143] It should be noted that each test device in the distributed testing system includes a test case execution module 502. This module provides an interface to facilitate the test task management module on the test device to query the current available resources of the test device.

[0144] Below, for reference Figure 6 This describes a computing device according to embodiments of the present application. Figure 6 The diagram shown is a structural schematic of a computing device provided in an embodiment of this application.

[0145] like Figure 6 As shown, the computing device includes one or more processors 601 and memory 602.

[0146] The processor 601 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the computing device to perform desired functions.

[0147] The memory 602 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 601 may execute the program instructions to implement functions such as the software testing methods of the various embodiments of this application mentioned above. Various contents such as model parameters and input data may also be stored in the computer-readable storage medium.

[0148] In one example, the computing device may also include an input device 603 and an output device 604, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0149] The input device 603 may include, for example, a keyboard, a mouse, etc.

[0150] The output device 604 can output various information to the outside, including inference results. The output device 604 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0151] Of course, for the sake of simplicity, Figure 6 Only some of the components of the computing device relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the computing device may include any other suitable components depending on the specific application.

[0152] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the software testing methods according to various embodiments of this application described above.

[0153] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the software testing methods according to various embodiments of this application described above.

[0155] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0156] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0157] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0158] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0159] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0160] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0161] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications or equivalent substitutions made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A software testing method, applied to a testing device, wherein, The method includes: In response to the test device receiving a test task request, the test task management thread determines the index of each test case included in the test task request based on the test task request; based on the index of each test case, it queries the preset test case information table to find the resources required for the test case with the corresponding index; Each test case and its required resource information are carried in the execution request corresponding to the test case, and the execution request corresponding to each test case is added to the test case waiting execution queue of the test equipment. The execution requests corresponding to the current test cases waiting to be executed in the test case waiting execution queue are obtained sequentially, and it is determined whether the current idle resources of the test device meet the resources required by the current test case; If the conditions are met, an execution thread is created to execute the current test case, and the current idle resources of the test device are updated; if the conditions are not met, the process waits for other execution threads to finish and release the occupied resources. Each execution thread executes test cases in parallel, and after the test cases are completed, the test results corresponding to the test cases and the index corresponding to the test cases are written into a preset test task information table. The test results include test success or test failure. When the test result is test success, the resources occupied by the test cases during runtime are written into the preset test case information table along with the index corresponding to the test cases. The test task waiting thread queries the preset test task information table. After all test cases in the test task request have been executed, a test report is generated based on the preset test task information table. The resources required for the test cases are set according to the maximum resources required to execute the test cases; The current idle resources are determined as follows: the current idle resources are obtained and recorded. The current idle resources are determined based on the total resource usage data of the test equipment, the over-issuance coefficient, and the resources occupied by the test cases being executed, wherein the over-issuance coefficient is greater than 1. Collect the real-time idle resources of the test equipment; if the collected real-time idle resources are less than the recorded current idle resources, then update the recorded current idle resources using the collected real-time idle resources and the over-issuance coefficient.

2. The method according to claim 1, wherein, If the current idle resources of the test device do not meet the resources required by the current test case, the method further includes: putting the current test case back to the tail of the test case waiting execution queue, and continuing to obtain the next execution request from the test case waiting execution queue.

3. The method according to claim 1, wherein, The required resources include at least one of the following: Required number of CPU cores, required number of GPU cores, required RAM capacity, required hard disk capacity, and required network bandwidth.

4. The method according to claim 1, wherein, The test device can be any test device in the distributed test system.

5. The method according to claim 4, wherein, After determining the resources required by each test case in the test case waiting execution queue based on the test task request, the method further includes: If the current idle resources of the test device do not meet the resource requirements of all test cases included in the test task request, then a portion of the test cases are split from the test cases included in the test task request and sent to other test devices; wherein the current idle resources of each test device to which test cases are assigned meet the resource requirements of the test cases to which they are assigned. For the remaining test cases after splitting, continue to execute the steps of carrying each test case and its required resource information in the execution request corresponding to the test case, and adding the execution request corresponding to each test case to the test case waiting execution queue of the test device.

6. A software testing apparatus, applied to testing equipment, wherein, The device includes: a test task management module and a test case execution module; wherein... The test task management module is configured to, in response to the test device receiving a test task request, use a test task management thread to determine the index of each test case included in the test task request based on the test task request; and query the resources required by the test case corresponding to the index from a preset test case information table based on the index of each test case. The test case execution module is configured to perform the following operations: Each test case and its required resource information are carried in the execution request corresponding to the test case, and the execution request corresponding to each test case is added to the test case waiting execution queue of the test equipment. The execution requests corresponding to the current test cases waiting to be executed in the test case waiting execution queue are obtained sequentially, and it is determined whether the current idle resources of the test device meet the resources required by the current test case; If the conditions are met, an execution thread is created to execute the current test case, and the current idle resources of the test device are updated; if the conditions are not met, the process waits for other execution threads to finish and release the occupied resources. Each execution thread executes test cases in parallel, and after the test cases are completed, the test results corresponding to the test cases and the index corresponding to the test cases are written into a preset test task information table. The test results include test success or test failure. When the test result is test success, the resources occupied by the test cases during runtime are written into the preset test case information table along with the index corresponding to the test cases. The test task waiting thread queries the preset test task information table. After all test cases in the test task request have been executed, a test report is generated based on the preset test task information table. The resources required for the test cases are set according to the maximum resources required to execute the test cases; The currently available free resources are recorded. The recorded currently available free resources are determined based on the total resource usage data of the test equipment, the over-issuance coefficient, and the resources occupied by the test cases being executed, wherein the over-issuance coefficient is greater than 1. Collect the real-time idle resources of the test equipment; if the collected real-time idle resources are less than the recorded current idle resources, then update the recorded current idle resources using the collected real-time idle resources and the over-issuance coefficient.

7. A computing device, characterized in that, Including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the method as described in any one of claims 1-5 by running a program in the memory.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method as described in any one of claims 1-5.

9. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, cause the processor to implement the method as described in any one of claims 1-5.

Images