A testing method and related equipment

Abstract

The present application discloses a testing method and related equipment, which includes: after receiving an automatic test request triggered by a user for a pending test task, first using the information index identifier of the test object carried by the automatic test request to query the test scheduling reference information corresponding to the information index identifier from a pre-built analysis information index database, so that the test scheduling reference information can indicate the minimum conditions required to be achieved when performing resource scheduling for the pending test task; then determining the resource configuration and task parameters of the pending test task based on the test scheduling reference information and the current state of the resource pool; finally, executing the pending test task according to the resource configuration and task parameters, thereby achieving the purpose of automatically executing the pending test task, thereby effectively avoiding the adverse effects caused by manually driving the test task, and further effectively reducing the resource consumption of the testing process.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a testing method and related equipment. Background Art

[0002] Currently, for some testing tasks, after the test cases are designed and reviewed, the testers perform the tests step by step according to the procedures described in the test cases to compare the actual results with the expected results.

[0003] However, since the above test solution is manually driven, it consumes a lot of resources (eg, human resources, etc.). Summary of the Invention

[0004] In order to solve the above technical problems, the present application provides a testing method and related equipment, which can effectively reduce the resource consumption of the testing process.

[0005] In order to achieve the above objectives, the technical solutions provided in the embodiments of the present application are as follows:

[0006] The present invention provides a testing method, which includes:

[0007] Receive an automatic test request triggered by a user for a pending test task; wherein the automatic test request carries an information index identifier of a test object of the pending test task;

[0008] Querying the test scheduling reference information corresponding to the information index identifier from a pre-built analysis information index database; wherein the test scheduling reference information is used to indicate the minimum conditions required to be met when scheduling resources for the pending test task;

[0009] Determining resource configuration of the pending test task and task parameters of the resource configuration according to the current state of the resource pool and the test scheduling reference information;

[0010] Execute the pending test task according to the resource configuration and the task parameters of the resource configuration.

[0011] In a possible implementation, the test scheduling reference information includes at least one of a low inflection point of the number of test task repetitions, a low inflection point of the tested service resource demand, and a maximum query rate per second (QPS) corresponding to the low inflection point of the tested service resource demand.

[0012] In a possible implementation, the test scheduling reference information includes a low inflection point of the number of test task repetitions, a low inflection point of the resource demand of the tested service, and a maximum QPS value corresponding to the low inflection point of the resource demand of the tested service;

[0013] The determining, based on the current state of the resource pool and the test scheduling reference information, the resource configuration of the to-be-processed test task and the task parameters of the resource configuration includes:

[0014] According to the current state of the resource pool and the low inflection point of the resource demand of the tested service, resource allocation processing is performed on the pending test task to obtain the resource configuration of the pending test task;

[0015] The task parameters of the resource configuration are determined according to the low inflection point of the number of repeated executions of the test task and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service.

[0016] In a possible implementation, performing resource allocation processing on the pending test task based on the current state of the resource pool and the low inflection point of the resource demand of the tested service to obtain the resource configuration of the pending test task includes:

[0017] Determining currently idle resources in the resource pool according to the current state of the resource pool;

[0018] According to the current idle resources and the low inflection point of resource demand of the tested service, resource allocation processing is performed on the pending test task to obtain resource configuration of the pending test task.

[0019] In one possible implementation, the method further includes:

[0020] Obtaining an estimated task density; wherein the estimated task density is used to represent the number of test tasks predicted to be executed simultaneously during the execution of the to-be-processed test task;

[0021] The performing resource allocation processing on the pending test task according to the current idle resources and the low inflection point of resource demand of the tested service to obtain the resource configuration of the pending test task includes:

[0022] According to the estimated task density, the current idle resources, and the low inflection point of the resource demand of the tested service, resource allocation processing is performed on the pending test task to obtain a resource configuration of the pending test task.

[0023] In a possible implementation, performing resource allocation processing on the pending test task based on the estimated task density, the current idle resources, and the low inflection point of resource demand of the tested service to obtain the resource configuration of the pending test task includes:

[0024] Determining resource stress level characterization data based on the estimated task density and the amount of the currently idle resources;

[0025] If the resource stress level characterization data meets the preset resource stress condition, resource allocation processing is performed on the pending test task from the current idle resources according to the low inflection point of the resource demand of the tested service to obtain the resource configuration of the pending test task.

[0026] In one possible implementation, the method further includes:

[0027] If the resource stress level characterization data does not meet the preset resource stress condition, querying the first increase information corresponding to the resource stress level characterization data at the low inflection point of the resource demand of the measured service from a pre-constructed first mapping relationship;

[0028] Determining the amount of resources to be used by using the first increase information and the low inflection point of the resource demand of the measured service;

[0029] According to the amount of resources to be used, resource allocation processing is performed on the test task to be processed from the current idle resources to obtain a resource configuration of the test task to be processed.

[0030] In a possible implementation, the estimated task density is obtained by prediction using execution description data of test tasks executed within a historical time period.

[0031] In a possible implementation, determining the task parameters of the resource configuration based on the low inflection point of the number of repeated executions of the test task and the maximum QPS corresponding to the low inflection point of the resource demand of the tested service includes:

[0032] If the resource configuration of the test task to be processed meets the preset minimum configuration conditions, the low inflection point of the number of repeated executions of the test task and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service are determined as the task parameters of the resource configuration.

[0033] In one possible implementation, the method further includes:

[0034] If the resource configuration of the pending test task meets the preset increase condition, the resource configuration of the pending test task is compared with the low inflection point of the resource demand of the tested service to obtain a resource abundance representation value;

[0035] Querying, from a pre-constructed second mapping relationship, second increase information corresponding to the resource abundance representation value at the maximum QPS value corresponding to the low inflection point of the number of repeated executions of the test task and the low inflection point of the resource demand of the tested service;

[0036] The task parameters of the resource configuration are determined by using the second adjustment information, the low inflection point of the number of repeated executions of the test task, and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service.

[0037] In a possible implementation, the method further includes: after determining that the pending test task is completed, updating the analysis information index database using the execution description data of the pending test task.

[0038] The present application also provides a testing device, including:

[0039] A receiving unit, configured to receive an automatic test request triggered by a user for a pending test task; wherein the automatic test request carries an information index identifier of a test object of the pending test task;

[0040] A query unit, configured to query the test scheduling reference information corresponding to the information index identifier from a pre-built analysis information index database; wherein the test scheduling reference information is used to indicate the minimum conditions required to be met when scheduling resources for the pending test task;

[0041] A determining unit, configured to determine the resource configuration of the to-be-processed test task and the task parameters of the resource configuration according to the current state of the resource pool and the test scheduling reference information;

[0042] The testing unit is used to execute the pending test task according to the resource configuration and the task parameters of the resource configuration.

[0043] An embodiment of the present application further provides a device comprising a processor and a memory: the memory is used to store a computer program; the processor is used to execute any implementation of the test method provided in the embodiment of the present application according to the computer program.

[0044] The embodiments of the present application further provide a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute any implementation of the test method provided in the embodiments of the present application.

[0045] The embodiments of the present application also provide a computer program product. When the computer program product is run on a terminal device, the terminal device executes any implementation of the testing method provided in the embodiments of the present application.

[0046] Compared with the prior art, the embodiments of the present application have at least the following advantages:

[0047] In the technical solution provided by the embodiment of the present application, after receiving an automatic test request triggered by a user for a pending test task, the information index identifier of the test object of the pending test task carried by the automatic test request is first used to query the test scheduling reference information corresponding to the information index identifier from a pre-built analysis information index database, so that the test scheduling reference information can represent the minimum conditions required to schedule resources for the pending test task; then, based on the test scheduling reference information and the current state of the resource pool, the resource configuration of the pending test task and the task parameters of the resource configuration are determined; finally, the pending test task is executed according to the resource configuration and the task parameters of the resource configuration, so that the purpose of automatically executing the pending test task can be achieved, thereby effectively avoiding the adverse effects caused by manually driven test tasks (for example, the need to consume a large amount of resources), and further effectively reducing the resource consumption of the testing process.

[0048] In addition, since the test scheduling reference information is used to indicate the minimum conditions that need to be met when scheduling resources for the test tasks to be processed (especially, the minimum amount of resources required to be used), the resource configuration and task parameters determined based on the test scheduling reference information can at least meet the minimum conditions, so that the resource configuration and task parameters can ensure the smooth execution of the test tasks to be processed. This can effectively avoid the defect that the test tasks to be processed cannot be executed or fail to execute due to insufficient resources scheduled for the test tasks to be processed, thereby improving the automatic execution effect of the test tasks to be processed.

[0049] In addition, since the resource configuration of the pending test task and the task parameters of the resource configuration are based on the current state of the resource pool, the "resource configuration of the pending test task and the task parameters of the resource configuration" are more in line with the current state of the resource pool. This can achieve the purpose of dispatching more resources to the pending test task when resources are abundant, and dispatching fewer resources to the pending test task when resources are scarce, which is conducive to improving the use effect of the resource pool (for example, utilization rate, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A flow chart of a testing method provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the engineering architecture of a test system provided in an embodiment of the present application;

[0053] Figure 3 A schematic structural diagram of a testing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In their research on test tasks, the inventors found that in some situations (for example, during peak periods of test tasks), the idle resources in the resource pool may not be able to meet the resource scheduling requirements of all test tasks. As a result, a large number of test tasks may not be able to be executed or may fail to execute due to insufficient resources allocated to them.

[0055] Based on the above findings, in order to solve the technical problems shown in the background technology part, an embodiment of the present application provides a testing method, which may specifically include: after receiving an automatic test request triggered by a user for a pending test task, first use the information index identifier of the test object of the pending test task carried by the automatic test request to query the test scheduling reference information corresponding to the information index identifier from a pre-built analysis information index database, so that the test scheduling reference information can represent the minimum conditions required to be achieved when performing resource scheduling for the pending test task; then, based on the test scheduling reference information and the current status of the resource pool, determine the resource configuration of the pending test task and the task parameters of the resource configuration; finally, execute the pending test task according to the resource configuration and the task parameters of the resource configuration, so that the purpose of automatically executing the pending test task can be achieved, thereby effectively avoiding the adverse effects caused by manually driven test tasks (for example, the need to consume a large amount of resources), and thus effectively reducing the resource consumption of the testing process.

[0056] In addition, since the test scheduling reference information is used to indicate the minimum conditions required to be achieved when scheduling resources for the test tasks to be processed (especially, the minimum amount of resources required to be used), the resource configuration and task parameters determined based on the test scheduling reference information can at least meet the minimum conditions, so that the resource configuration and task parameters can ensure the smooth execution of the test tasks to be processed. This can effectively avoid the phenomenon that the test tasks to be processed cannot be executed or fail to execute due to insufficient resources scheduled for the test tasks to be processed, thereby improving the automatic execution effect of the test tasks to be processed.

[0057] Furthermore, the embodiments of the present application do not limit the execution subject of the test method. For example, the test method provided in the embodiments of the present application can be applied to data processing devices such as terminal devices or servers. The terminal device can be a smartphone, a computer, a personal digital assistant (PDA), or a tablet computer. The server can be a standalone server, a cluster server, or a cloud server.

[0058] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0059] Method Example 1

[0060] See also Figure 1 , which is a flow chart of a testing method provided in an embodiment of the present application.

[0061] The test method provided in the embodiment of the present application includes S1-S4:

[0062] S1: Receive an automatic test request triggered by a user for a pending test task.

[0063] Here, the user refers to the trigger of the automatic test request; and the embodiment of the present application does not limit the user. For example, the user may be a relevant person of the test task to be processed (for example, a tester, a reviewer, a programmer, etc.).

[0064] The pending test task is used to perform functional testing on a certain service under test (for example, XXX software, YYY application, etc.). In other words, the aforementioned "certain service under test" is the test object of the pending test task.

[0065] It should be noted that the embodiments of the present application are not limited to the above-mentioned “pending test tasks”; and the embodiments of the present application are not limited to the above-mentioned “certain tested service”.

[0066] The automatic test request is used to request automatic execution of a pending test task; and the automatic test request carries an information index identifier of a test object of the pending test task.

[0067] The above-mentioned “information index identifier” is used to uniquely identify the test object of the test task to be processed, so that the “information index identifier” can be used to query some information related to the test object (for example, the “test scheduling reference information” below) later.

[0068] In addition, the embodiment of the present application is not limited to the above-mentioned "information index identifier", for example, it can be the name of the test object of the test task to be processed. For another example, the "information index identifier" can be obtained by performing a numerical conversion process on the name of the test object of the test task to be processed.

[0069] It should be noted that the embodiments of the present application do not limit the implementation method of the numerical conversion processing. For example, it can be implemented using any existing or future method that can convert text into numerical values ​​(for example, word2vec).

[0070] Based on the relevant content of S1 above, it can be known that when the user wants to automatically execute the pending test task, the user can trigger an automatic test request for the pending test task on the test device, so that the test device can automatically perform resource scheduling processing and task execution processing for the pending test task based on the information index identifier carried by the automatic test request, thereby realizing the automatic testing process of the test object (for example, XXX software, etc.) of the pending test task.

[0071] It should be noted that the above-mentioned "test device" refers to any data processing device (for example, a terminal device or a server) used to execute the test method provided in the embodiments of the present application.

[0072] S2: Query the test scheduling reference information corresponding to the information index identifier from a pre-built analysis information index database.

[0073] The analysis information index database is used to record a large amount of test scheduling candidate information corresponding to candidate index identifiers.

[0074] In addition, the embodiments of the present application do not limit the above-mentioned "analysis information index database". For example, it may specifically include the 1st candidate index identifier and the 1st test scheduling candidate information, the 2nd candidate index identifier and the 2nd test scheduling candidate information, ..., and the Kth candidate index identifier and the Kth test scheduling candidate information. Among them, the kth candidate index identifier is used to uniquely represent the kth candidate tested service (for example, ZZZ software, etc.). The kth test scheduling candidate information is used to represent the minimum conditions that need to be achieved when performing functional testing on the kth candidate tested service (that is, when executing a test task for performing functional testing on the kth candidate tested service). k is a positive integer, k≤K, and K is a positive integer.

[0075] It should be noted that the relevant content of the above-mentioned “k-th test scheduling candidate information” is similar to the relevant content of the following “test scheduling reference information”.

[0076] It can be seen that after obtaining the information index identifier of the test object of the test task to be processed, the information index identifier can be matched with the K candidate index identifiers in the analysis information index database, so that when it is determined that the information index identifier successfully matches the v-th candidate index identifier, the v-th test scheduling candidate information corresponding to the v-th candidate index identifier can be directly determined as the test scheduling reference information corresponding to the information index identifier. Wherein, v is a positive integer, v∈{1, 2, ..., K}.

[0077] In addition, the present application embodiment does not limit the construction process of the above-mentioned "analysis information index database", please refer to the following Method Example 2 The relevant content of the above-mentioned "analysis information index database" is shown.

[0078] The above-mentioned “test scheduling reference information” is used to indicate the minimum conditions required to be met when scheduling resources for pending test tasks (for example, the minimum number of repetitions required for pending test tasks and the minimum resource requirements of pending test tasks, etc.).

[0079] In addition, the embodiments of the present application do not limit the above-mentioned "test scheduling reference information". For example, it may specifically include at least one of the low inflection point of the number of repeated executions of the test task, the low inflection point of the resource demand of the service under test, and the maximum query rate per second (QPS, Queries-per-second) corresponding to the low inflection point of the resource demand of the service under test.

[0080] The above-mentioned "low inflection point of the number of repeated executions of the test task" refers to the minimum value that the number of repeated executions of the test task to be processed needs to reach, so that the test task to be processed can just meet the test validity conditions such as test indicator stability and test result confidence at the "low inflection point of the number of repeated executions of the test task". In this way, the validity of the test task can be guaranteed, thereby avoiding the occurrence of invalid test.

[0081] It can be seen that if the number of repeated executions actually used when executing the pending test task is lower than the above-mentioned "low inflection point of the number of repeated executions of the test task", the actual test results for the pending test task will not be able to meet the test result confidence requirements, thereby resulting in invalid testing; however, if the number of repeated executions actually used when executing the pending test task is equal to or higher than the "low inflection point of the number of repeated executions of the test task", the actual test results for the pending test task will be able to meet the test result confidence requirements, thereby ensuring the effectiveness of the test task.

[0082] Based on the above two paragraphs, it can be seen that when automatically executing a pending test task, the pending test task can be executed repeatedly multiple times, so that subsequent analysis can be performed based on the multiple execution data of the pending test task (for example, execution time, whether the task is completed, actual resource usage, etc.). This can effectively avoid the adverse effects caused by accidental events, thereby helping to improve the automatic execution effect of the pending test task.

[0083] The above-mentioned "low inflection point of resource demand of the service under test" is used to represent the minimum resource demand of the service under test (that is, the test object of the test task to be processed), so that the service under test can just meet the conditions such as the service under test can be started normally and can stably and continuously receive traffic at the "low inflection point of resource demand of the service under test". This can ensure the effectiveness of the environment and effectively avoid the occurrence of environmental unavailability.

[0084] It can be seen that if the actual amount of resources allocated to the test task to be processed is lower than the above-mentioned "low inflection point of resource demand for the service under test", the environment will become unavailable due to the inability to meet the resource boundary value of the availability of the service under test, thereby causing the test task to fail to execute; however, if the actual amount of resources allocated to the test task to be processed is equal to or greater than the "low inflection point of resource demand for the service under test", the resource boundary value of the availability of the service under test can be met, so that the test object of the test task to be processed can be started normally and can receive traffic stably and continuously, thereby ensuring the environmental effectiveness of the test task to be processed.

[0085] The "maximum QPS corresponding to the low inflection point of resource demand for the tested service" is used to represent the maximum QPS that can be undertaken under the minimum resource demand of the tested service (that is, the test object of the test task to be processed), to ensure that the environment is not unavailable due to excessive test QPS. It should be noted that in this application, QPS can be understood as the number of test cases executed per second.

[0086] It can be seen that when the resources actually allocated for the pending test tasks are equal to the above-mentioned "low inflection point of resource demand for the tested service", if the QPS actually used when executing the pending test tasks is higher than the above-mentioned "maximum QPS corresponding to the low inflection point of resource demand for the tested service", the test QPS is too large, resulting in insufficient resources for the above-mentioned "resources actually allocated for the pending test tasks", thereby causing the environment to be unavailable, and then causing the test task execution to fail; if the QPS actually used when executing the pending test tasks is equal to the "maximum QPS corresponding to the low inflection point of resource demand for the tested service", not only can the environment availability be maintained, but the maximum task concurrency can also be achieved, thereby effectively improving the test efficiency; if the QPS actually used when executing the pending test tasks is lower than the above-mentioned "maximum QPS corresponding to the low inflection point of resource demand for the tested service", although the environment availability can be maintained, it will lead to insufficient utilization of resources.

[0087] Based on the relevant content of S2 above, it can be known that after obtaining the information index identifier of the test object of the test task to be processed, the test scheduling reference information corresponding to the information index identifier can be queried from the pre-built analysis information index database, so that the test scheduling reference information can represent the minimum conditions required to be achieved when scheduling resources for the test task to be processed (for example, the minimum number of repeated executions of the test task, the minimum resource requirement, and the maximum QPS that can be tolerated under the minimum resource requirement, etc.), so that the resources allocated based on the test scheduling reference information can meet the minimum conditions, thereby effectively avoiding the phenomenon of failure of execution of the test task to be processed due to insufficient allocated resources, thereby effectively reducing the possibility of failure of execution of the test task to be processed, and thus improving the automatic execution effect of the test task to be processed.

[0088] S3: Determine the resource configuration of the test task to be processed and the task parameters of the resource configuration according to the current state of the resource pool and the test scheduling reference information.

[0089] The resource pool is used to provide the resources required by the test task to be processed. Furthermore, the present embodiment is not limited to the resource pool. For example, the resource pool may include at least one cluster, at least one server, or at least one container.

[0090] The above-mentioned “current status of the resource pool” is used to indicate the real-time usage of the resource pool (for example, which resources are in use, which resources are in idle state, etc.).

[0091] Furthermore, the acquisition time of the "current state of the resource pool" is later than the reception time of the automatic test request and earlier than the execution time of S3. Furthermore, the present embodiment does not limit the acquisition time of the "current state of the resource pool." Furthermore, the present embodiment does not limit the acquisition method of the "current state of the resource pool."

[0092] The above-mentioned “resource configuration of the pending test task” is used to indicate the resources allocated to the pending test task.

[0093] The above-mentioned “task parameters of resource configuration” are used to represent some parameters related to the execution of the test task (for example, the maximum acceptable QPS of the resource configuration and the number of repeated executions of the task, etc.) regulated by the above-mentioned “resource configuration of the test task to be processed”.

[0094] In addition, the embodiments of the present application do not limit the above-mentioned "task parameters of resource configuration", for example, it may include: at least one of the maximum acceptable QPS of the resource configuration and the maximum acceptable number of task repetitions of the resource configuration.

[0095] In order to better understand the above “resource configuration of the test task to be processed” and “task parameters of the resource configuration”, a possible implementation of S3 is used as an example for explanation below.

[0096] As an example, when the above "test scheduling reference information" includes the low inflection point of the number of test task repetitions, the low inflection point of the resource demand of the tested service, and the maximum QPS corresponding to the low inflection point of the resource demand of the tested service, S3 may specifically include S31-S32:

[0097] S31: Based on the current state of the resource pool and the low inflection point of the resource demand of the service under test, resource allocation processing is performed on the test task to be processed to obtain the resource configuration of the test task to be processed so that the resource amount of the resource configuration is not lower than the low inflection point of the resource demand of the service under test.

[0098] In an embodiment of the present application, after obtaining the current state of the resource pool and the low inflection point of the resource demand of the service under test, these two pieces of information can be used as reference to perform resource allocation processing on the test task to be processed, and obtain the resource configuration of the test task to be processed, so that the resource configuration can adapt to the current state of the resource pool as much as possible while ensuring that its resource amount is not lower than the low inflection point of the resource demand of the service under test. This can effectively avoid the phenomenon of failure of execution of the test task to be processed due to insufficient resources allocated to the test task to be processed.

[0099] In addition, the embodiment of the present application does not limit the implementation of S31. For ease of understanding, two possible implementations are described below.

[0100] In a first possible implementation, S31 may specifically include S311-S312:

[0101] S311: Determine the current idle resources in the resource pool according to the current state of the resource pool.

[0102] The currently idle resources are used to indicate the resources in the resource pool that are currently idle.

[0103] In addition, the embodiment of the present application does not limit the implementation method of S311. For example, it can be implemented by using any existing or future idle resource determination method.

[0104] In addition, the embodiment of the present application does not limit the execution time of S311, as long as the execution time of S311 is ensured to be earlier than S312.

[0105] S312: performing resource allocation processing on the test task to be processed according to the current idle resources in the resource pool and the low inflection point of the resource demand of the tested service to obtain the resource configuration of the test task to be processed.

[0106] In an embodiment of the present application, after obtaining the current idle resources of the resource pool, the low inflection point of the resource demand of the service under test can be referred to, and resource allocation processing can be performed on the test task to be processed from the current idle resources to obtain the resource configuration of the test task to be processed, so that the resource configuration can use the current idle resources of the resource pool as much as possible while ensuring that its resource amount is not lower than the low inflection point of the resource demand of the service under test. This can effectively avoid the phenomenon of failure of execution of the test task to be processed due to insufficient resources allocated to the test task to be processed.

[0107] Based on the relevant contents of S311 to S312 above, it can be known that after obtaining the current state of the resource pool, the current idle resources of the resource pool can be determined based on the current state; then, based on the low inflection point of the resource demand of the service under test, the resource allocation processing of the test task to be processed is performed from the current idle resources to obtain the resource configuration of the test task to be processed, so that the resource configuration can use the current idle resources of the resource pool as much as possible while ensuring that its resource amount is not lower than the low inflection point of the resource demand of the service under test. This can effectively avoid the phenomenon of failure of the test task to be processed due to insufficient resources allocated to the test task to be processed.

[0108] In fact, in some application scenarios, a large number of test tasks are usually executed simultaneously in the same time period. Based on this, the embodiment of the present application also provides a second possible implementation method of S31, which may specifically include steps 11 and 12:

[0109] Step 11: According to the current state of the resource pool, determine the current idle resources of the resource pool.

[0110] It should be noted that for the relevant content of step 11, please refer to S311 above.

[0111] In addition, the embodiment of the present application does not limit the execution time of step 11, as long as the execution time of step 11 is ensured to be earlier than that of step 12.

[0112] Step 12: Based on the estimated task density, current idle resources, and the low inflection point of resource demand for the tested service, resource allocation is performed on the pending test task to obtain the resource configuration of the pending test task.

[0113] The estimated task density is used to indicate the number of test tasks to be executed by the predicted resource pool, especially the number of test tasks to be executed simultaneously during the execution of the pending test tasks.

[0114] In addition, the estimated task density can be predicted using the execution description data of test tasks executed during the historical time period. The execution description data describes the relevant information generated during the execution of a test task (for example, whether the test task was completed, the resource configuration, the QPS controlled under the resource configuration, the execution time period, the execution duration, the number of task repetitions, etc.).

[0115] It should be noted that the embodiments of the present application are not limited to the above-mentioned "historical time period". For example, it can be one month (or even longer) before the triggering time point of the above-mentioned "automatic test request". In addition, the embodiments of the present application are not limited to the above-mentioned "estimated task density" prediction process. For example, any existing or future prediction analysis method (for example, a method based on a machine learning model, a big data mining method, etc.) can be used for implementation.

[0116] In addition, the embodiment of the present application does not limit the acquisition time of the above-mentioned "estimated task density", as long as it is ensured to be completed before executing step 12.

[0117] The embodiment of the present application does not limit the implementation of step 12. For example, it may specifically include steps 121 to 126:

[0118] Step 121: Determine resource stress level representation data based on the estimated task density and the current amount of idle resources.

[0119] Among them, the resource tension characterization data is used to represent the resource tension level of the current idle resources in the resource pool under the estimated task density, so that the "resource tension characterization data" can reflect whether the current idle resources can meet the test task resource requirements under the estimated task density.

[0120] In addition, the embodiment of the present application does not limit the determination process of the above-mentioned "resource tension characterization data". For example, resource tension characterization data = resource amount of current idle resources ÷ (estimated task density × average task resource demand). Among them, the average task resource demand is used to represent the average amount of resources required for the test task.

[0121] It should be noted that the embodiments of the present application do not limit the method for obtaining the above-mentioned "average task resource requirement". For example, it can be set first. For another example, it can also be: performing big data analysis and processing on the execution description data of the test tasks executed in the historical time period to obtain the average task resource requirement.

[0122] Step 122: Determine whether the resource stress level characterizing data satisfies a preset resource stress condition. If so, execute the following step 123; if not, execute the following steps 124 to 126.

[0123] The preset resource strain condition may be pre-set. For example, when the resource strain degree representation data is greater, it indicates that the current idle resources in the resource pool are less likely to meet the resource scheduling requirements of the test task under the estimated task density, thereby indicating that the resources are more strained. The preset resource strain condition may be: the resource strain degree representation data is greater than a first threshold.

[0124] Based on the relevant content of step 122, it can be known that after obtaining the data representing the degree of resource tension, it is determined whether the data representing the degree of resource tension meets the preset resource tension condition. If the preset resource tension condition is met, it means that the current idle resources in the resource pool are resource-tight relative to the estimated task density. Therefore, in order to ensure that as many test tasks as possible are effectively executed, resource scheduling can be performed according to the minimum resource configuration of each test task (that is, the resource allocation method shown in step 123). This can effectively ensure that as many test tasks as possible can be effectively executed; however, if it is not met, it means that the current idle resources in the resource pool are not tight relative to the estimated task density. Therefore, in order to better complete the test tasks, resource scheduling can be performed according to the better resource configuration of each test task (that is, the resource allocation method shown in steps 124-126). This can not only ensure that these test tasks are completed smoothly, but also ensure that these tasks are completed with higher quality (or faster speed), which is conducive to improving the automatic execution effect of these test tasks.

[0125] Step 123: According to the low inflection point of the resource demand of the service under test, resource allocation processing is performed on the pending test task from the current idle resources to obtain the resource configuration of the pending test task.

[0126] In an embodiment of the present application, when it is determined that the data characterizing the degree of resource tension meets the preset resource tension condition, it can be determined that the current idle resources of the resource pool are relatively tight relative to the estimated task density. Therefore, the resource allocation processing can be performed directly on the test task to be processed from the current idle resources according to the low inflection point of the resource demand of the service to be tested, and the resource configuration of the test task to be processed is obtained, so that the resource amount of the resource configuration is equal to the low inflection point of the resource demand of the service to be tested, so that the test task to be processed can just meet the test validity requirements of test indicator stability and test result confidence under the resource configuration. In this way, it is possible to reduce resource consumption as much as possible while ensuring that the test task to be processed can be effectively executed, thereby effectively ensuring that a large number of test tasks are effectively executed as much as possible during the execution of the test task to be processed.

[0127] Step 124: querying the first increase information corresponding to the resource intensity representation data at the low inflection point of the resource demand of the measured service from the pre-built first mapping relationship.

[0128] Among them, the first mapping relationship is used to record a large number of candidate adjustment information corresponding to the resource tension characterization values ​​at various low inflection points of alternative service resource requirements; and the embodiment of the present application does not limit the first mapping relationship, for example, it may include the corresponding relationship shown in Table 1.

[0129]

[0130] Table 1 First mapping relationship

[0131] It should be noted that, in Table 1, A represents the number of resource stress characterization values; and B represents the number of low-level inflection points of resource demand for alternative services.

[0132] Based on the relevant content of the first mapping relationship above, it can be known that after obtaining the resource tension representation data, the to-be-used tuple (resource tension representation data, low inflection point of resource demand for the measured service) can be matched with each candidate tuple (resource tension representation value, low inflection point of resource demand for the alternative service) in the first mapping relationship, so as to determine that the resource tension representation data is equal to the hth resource tension representation value a. h , and determine that the low inflection point of the resource demand of the service being tested is equal to the low inflection point b of the resource demand of the gth candidate service g When the corresponding relationship R h,g The candidate adjustment information C recorded in h,g , which is determined as the first increase information corresponding to the resource intensity representation data at the low inflection point of the measured service resource demand. Wherein, h is a positive integer, h∈{1, 2, 3, ..., A}; g is a positive integer, g∈{1, 2, 3, ..., B}.

[0133] In addition, the embodiment of the present application does not limit the method for obtaining the first mapping relationship, which may be pre-set. In another example, it may be: performing big data analysis on the execution description data of the test tasks executed in the historical time period to obtain the first mapping relationship.

[0134] The above-mentioned "first adjustment information" is used to indicate the relative relationship between the actual amount of resources allocated for the test task to be processed and the low inflection point of the resource demand of the service under test; and the embodiment of the present application does not limit the "first adjustment information", for example, it can be a ratio or an increase in resource amount.

[0135] Step 125: Determine the amount of resources to be used by using the first increase information and the low inflection point of the measured service resource demand.

[0136] The amount of resources to be used is used to represent the amount of resources actually allocated for the test tasks to be processed; and the amount of resources to be used is not less than the "low inflection point of resource demand for the tested service" mentioned above.

[0137] In addition, the embodiment of the present application does not limit the process of determining the amount of resources to be used (that is, the implementation method of step 125). For ease of understanding, two examples are used below for illustration.

[0138] Example 1: When the above-mentioned "first adjustment information" is used to represent the proportional relationship between the amount of resources actually allocated for the test task to be processed and the low inflection point of the resource demand of the service under test, step 125 can specifically be: multiplying the first adjustment information by the low inflection point of the resource demand of the service under test to obtain the amount of resources to be used.

[0139] Example 2: When the above-mentioned "first adjustment information" is used to represent the difference between the actual amount of resources allocated for the test task to be processed and the low inflection point of the resource demand of the service under test, step 125 can specifically be: adding the first adjustment information and the low inflection point of the resource demand of the service under test to obtain the amount of resources to be used.

[0140] Step 126: According to the amount of resources to be used, resource allocation processing is performed on the pending test task from the current idle resources to obtain the resource configuration of the pending test task.

[0141] In an embodiment of the present application, after obtaining the amount of resources to be used, resource allocation processing can be performed on the test task to be processed from the current idle resources according to the amount of resources to be used to obtain the resource configuration of the test task to be processed, so that the resource amount of the resource configuration is equal to the amount of resources to be used, so that the test task to be processed can not only be effectively executed under the resource configuration, but also can be executed with the best possible execution effect (for example, the fastest possible execution speed, etc.), which is conducive to improving the automatic execution effect of the test task to be processed.

[0142] Based on the relevant contents of the above steps 121 to 126, it can be known that after obtaining the estimated task density and the current idle resources, it is possible to first determine whether the resources are tight based on the estimated task density and the current idle resources; if it is determined that the resources are tight, the resource allocation processing can be performed directly on the test task to be processed from the current idle resources according to the low inflection point of the resource demand of the service to be tested, and the resource configuration of the test task to be processed can be obtained, so that the test task to be processed can be subsequently executed according to the minimum resource configuration, thereby achieving the goal of reducing resource consumption as much as possible while ensuring that the test task to be processed can be effectively executed, thereby effectively ensuring that a large number of test tasks are effectively executed as much as possible during the execution of the test task to be processed; however, if it is determined that the resources are not tight, the amount of resources to be used can be determined first, so that the amount of resources to be used is higher than the low inflection point of the resource demand of the service to be tested; then according to the amount of resources to be used, the resource allocation processing can be performed on the test task to be processed from the current idle resources, and the resource configuration of the test task to be processed can be obtained, thereby improving the automatic execution effect of the test task to be processed (for example, improving the automatic execution efficiency, etc.).

[0143] Based on the relevant content of the second possible implementation method of S31 above, it can be known that after obtaining the current state of the resource pool, you can first refer to the current state and the estimated task density to determine the degree of resource tension; then, according to the resource allocation method applicable to the degree of resource tension, perform resource allocation processing on the test task to be processed, and obtain the resource configuration of the test task to be processed, so that the resource configuration can improve the automatic execution effect of the test task to be processed as much as possible while ensuring the effective execution of the test task to be processed, so that intelligent resource allocation can be achieved for the test tasks to be processed according to different degrees of resource tension.

[0144] S32: Determine the task parameters for resource allocation according to the low inflection point of the number of repeated executions of the test task and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service.

[0145] As an example, S32 may specifically include S321-S326:

[0146] S321: Determine whether the resource configuration of the test task to be processed meets the preset minimum configuration conditions. If so, execute S322; if not, execute S323.

[0147] Among them, the preset minimum configuration condition refers to the condition satisfied by the test task allocated with the minimum resource configuration; and the embodiment of the present application does not limit the preset minimum configuration condition. For example, it can specifically be: the amount of resources is equal to the above "low inflection point of the number of repeated executions of the test task".

[0148] It can be seen that after obtaining the resource configuration of the test task to be processed, it can be determined whether the resource amount of the resource configuration is equal to the above "low inflection point of the number of repeated executions of the test task". If it is equal, it means that the resource configuration is the minimum resource configuration of the test task to be processed, so it can be determined that the resource configuration meets the preset minimum configuration conditions, so the task parameter control method corresponding to the preset minimum configuration conditions (as shown in S322 below) can be directly used for control.

[0149] S322: Determine the maximum QPS value corresponding to the low inflection point of the number of repeated executions of the test task and the low inflection point of the resource demand of the tested service as task parameters for resource configuration.

[0150] In an embodiment of the present application, when it is determined that the resource configuration of the test task to be processed meets the preset minimum configuration conditions, it can be determined that the minimum resource configuration method is adopted for the test task to be processed. Therefore, in order to adapt to the minimum resource configuration method as much as possible, the maximum acceptable QPS and the number of task repetitions under the "resource configuration of the test task to be processed" can be directly adjusted according to the low inflection point of the number of repeated executions of the test task and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service, to obtain the task parameters of the resource configuration, so that the maximum acceptable QPS recorded in the task parameters is the above-mentioned "maximum QPS value corresponding to the low inflection point of the resource demand of the tested service", and the maximum acceptable number of task repetitions recorded in the task parameters is the above-mentioned "low inflection point of the number of repeated executions of the test task". In this way, the automatic execution effect of the test task to be processed can be improved as much as possible while ensuring the effective execution of the test task to be processed.

[0151] S323: Determine whether the resource configuration of the pending test task meets the preset increase condition. If so, execute S324-S326; if not, generate and send a prompt message.

[0152] Among them, the preset increase condition refers to the condition satisfied by the test task that is allocated a higher resource configuration (that is, higher than the minimum resource configuration); and the embodiment of the present application does not limit the preset increase condition, for example, it can specifically be: the amount of resources is greater than the above "low inflection point of the number of repeated executions of the test task".

[0153] It can be seen that after obtaining the resource configuration of the test task to be processed, it can be determined whether the resource amount of the resource configuration is greater than the above "low inflection point of the number of repeated executions of the test task". If it is greater, it can be determined that the resource configuration meets the preset increase condition, so the task parameter control method corresponding to the preset increase condition can be directly adopted for subsequent control (as shown in S324-S326 below); however, if it is less than, it can be determined that the resource configuration neither meets the preset minimum configuration condition nor the preset increase condition, so it can be determined that there is a defect in the resource configuration (for example, insufficient resources, etc.), and it can be inferred that the resource configuration is likely to cause the execution of the test task to be processed to fail, so a prompt message can be directly generated and sent, so that the prompt message can inform the user that there is a problem with the resource configuration allocated to the test task to be processed, so that the user can be aware of this abnormal situation based on the prompt message, and then the user can take corresponding handling measures for this abnormal situation (for example, manually end the execution process of the test task to be processed in a timely manner; or, record the test task to be processed, etc.).

[0154] S324: Compare the resource configuration of the test task to be processed with the low inflection point of the resource demand of the tested service to obtain a resource abundance representation value.

[0155] The resource abundance representation value is used to indicate the relative relationship between the amount of resources actually allocated to the test task to be processed and the low inflection point of the resource demand of the tested service.

[0156] In addition, the embodiments of the present application do not limit the determination process of the above-mentioned "resource abundance characterization value". For example, it can be specifically: the ratio between the resource amount of the resource configuration of the test task to be processed and the low inflection point of the resource demand of the service under test is determined as the resource abundance characterization value.

[0157] S325: Query the second increase information corresponding to the resource abundance representation value at the QPS maximum value corresponding to the low inflection point of the test task repetition count and the low inflection point of the resource demand of the tested service from the pre-built second mapping relationship.

[0158] The second mapping relationship is used to record the corresponding information for the increase in resource abundance value under various combinations of (low inflection points of candidate task repetition times, candidate QPS values) for a large number of resources. Furthermore, the present embodiment does not limit this second mapping relationship; for example, the second mapping relationship may be similar to the "first mapping relationship" described above. That is, the second mapping relationship may include the corresponding relationships shown in Table 2.

[0159]

[0160] Table 2 Second mapping relationship

[0161] It should be noted that, for Table 2, D represents the number of resource abundance values; E represents the number of low-order inflection points of the number of repeated executions of candidate tasks; and F represents the number of candidate QPS values.

[0162] Based on the relevant content of the above second mapping relationship, it can be known that after obtaining the resource abundance representation value, the triplet to be used (resource abundance representation value, low inflection point of the number of repeated executions of the test task, and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service) can be matched with each candidate triplet in the second mapping relationship (resource abundance value, low inflection point of the number of repeated executions of the alternative task, and candidate QPS value) to determine whether the resource abundance representation value is equal to the wth resource abundance value d w The above “low inflection point of repeated execution times of the test task” is equal to the low inflection point of repeated execution times of the qth candidate task, e q , and when the above “maximum QPS value corresponding to the low inflection point of the measured service resource demand” is equal to the pth candidate QPS value f1, the corresponding relationship R w,q,p The candidate adjustment information G recorded in w,q,p , which is determined as the second height adjustment information. Wherein, w is a positive integer, w∈{1, 2, 3, ..., D}; q is a positive integer, q∈{1, 2, 3, ..., E}; and p is a positive integer, p∈{1, 2, 3, ..., F}.

[0163] In addition, the embodiment of the present application does not limit the method for obtaining the second mapping relationship, which may be pre-set. In another example, it may be: performing big data analysis on the execution description data of the test tasks executed in the historical time period to obtain the second mapping relationship.

[0164] The second adjustment information is used to represent the relative relationship between the task parameters actually controlled for the above-mentioned "resource configuration of the test tasks to be processed" and the tuple (the low inflection point of the number of repeated executions of the test task, and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service); and the embodiment of the present application does not limit the representation method of the second adjustment information, for example, it can be represented by a ratio or by a numerical increase.

[0165] S326: Determine the task parameters for resource allocation using the second adjustment information, the low inflection point of the number of repeated executions of the test task, and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service.

[0166] As an example, when the second increase information includes execution count increase information and QPS increase information, S326 may specifically include S3261-S3263:

[0167] S3261: Determine the number of cycles of the task to be referenced based on the execution count increase information and the low inflection point of the test task repetition count.

[0168] Among them, the execution count adjustment information is used to indicate the relative relationship between the maximum acceptable number of task repetitions actually regulated for the above-mentioned "resource configuration of test tasks to be processed" and the above-mentioned "low inflection point of the number of test task repetitions"; and the embodiment of the present application does not limit the execution count adjustment information, for example, it can be a proportional value or an increase in the number of times.

[0169] The number of cycles of the pending reference task is used to represent the maximum acceptable number of repeated executions of the task actually regulated by the above-mentioned “resource configuration of the pending test task”.

[0170] In addition, the process of determining the “number of task cycles to be referenced” is similar to the process of determining the amount of resources to be used shown in step 125 above.

[0171] S3262: Determine a reference QPS value based on the QPS increase information and the maximum QPS value corresponding to the low inflection point of the resource demand of the measured service.

[0172] Among them, the QPS increase information is used to indicate the relative relationship between the maximum acceptable QPS actually regulated for the above-mentioned "resource configuration of the test task to be processed" and the above-mentioned "maximum QPS value corresponding to the low inflection point of the resource demand of the tested service"; and the embodiment of the present application does not limit the QPS increase information, for example, it can be a proportional value or an increase in the number of times.

[0173] The reference QPS value is used to represent the maximum acceptable QPS actually regulated for the above-mentioned “resource configuration of the test task to be processed”.

[0174] In addition, the process of determining the “reference QPS value” is similar to the process of determining the amount of resources to be used shown in step 125 above.

[0175] S3263: According to the number of reference task cycles and the reference QPS value, perform task parameter control processing on the resource configuration of the test task to be processed to obtain the task parameters of the resource configuration.

[0176] In an embodiment of the present application, after obtaining the number of task cycles to be referenced and the QPS value to be referenced, the maximum acceptable number of task repetitions and its QPS under the above-mentioned "resource configuration of the test task to be processed" can be adjusted according to the two, and the task parameters of the resource configuration can be obtained, so that the maximum acceptable QPS recorded in the task parameters is the above-mentioned "QPS value to be referenced", and the maximum acceptable number of task repetitions recorded in the task parameters is the above-mentioned "number of task cycles to be referenced". In this way, the automatic execution effect of the test task to be processed can be improved as much as possible while ensuring the effective execution of the test task to be processed.

[0177] Based on the relevant contents of S3261 to S3263 above, it can be known that after obtaining the second adjustment information, the second adjustment information, the low inflection point of the number of repeated executions of the test task, and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service can be combined to determine the task parameters of the resource configuration so that the task parameters meet the adjustment requirements expressed by the second adjustment information.

[0178] Based on the relevant content of S32 above, it can be known that after obtaining the low inflection point of the number of repeated executions of the test task and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service, these two pieces of information can be referred to to determine the task parameters of the above-mentioned "resource configuration of the test task to be processed" so that the task parameters can be adapted to the "resource configuration of the test task to be processed" as much as possible, which is conducive to ensuring better execution of the test task to be processed.

[0179] Based on the relevant content of S3 above, it can be known that after obtaining the test scheduling reference information, the test scheduling reference information and the current status of the resource pool can be referred to to perform resource configuration processing and task parameter regulation processing on the test task to be processed, and obtain the resource configuration of the test task to be processed and the task parameters of the resource configuration.

[0180] S4: Execute the test task to be processed according to the resource configuration of the test task to be processed and the task parameters of the resource configuration.

[0181] In an embodiment of the present application, after obtaining the resource configuration of the test task to be processed and the task parameters of the resource configuration, the test environment of the test task to be processed can be deployed according to the resource configuration; then, in the test environment, the test task to be processed can be executed according to the task parameters of the resource configuration, thereby achieving the purpose of automatically executing the test task to be processed.

[0182] Based on the relevant contents of S1 to S4 above, it can be seen that for the test method provided in the embodiment of the present application, after receiving the automatic test request triggered by the user for the pending test task, the information index identifier of the test object of the pending test task carried by the automatic test request is first used to query the test scheduling reference information corresponding to the information index identifier from the pre-built analysis information index database, so that the test scheduling reference information can represent the minimum conditions required to be achieved when performing resource scheduling for the pending test task; then, based on the test scheduling reference information and the current status of the resource pool, the resource configuration of the pending test task and the task parameters of the resource configuration are determined; finally, the pending test task is executed according to the resource configuration and the task parameters of the resource configuration, so that the purpose of automatically executing the pending test task can be achieved, thereby effectively avoiding the adverse effects caused by manually driven test tasks (for example, the need to consume a large amount of resources), and thus effectively reducing the resource consumption of the testing process.

[0183] In addition, since the test scheduling reference information is used to indicate the minimum conditions that need to be met when scheduling resources for the test tasks to be processed (especially, the minimum amount of resources required to be used), the resource configuration and task parameters determined based on the test scheduling reference information can at least meet the minimum conditions, so that the resource configuration and task parameters can ensure the smooth execution of the test tasks to be processed. This can effectively avoid the defect that the test tasks to be processed cannot be executed or fail to execute due to insufficient resources scheduled for the test tasks to be processed, thereby improving the automatic execution effect of the test tasks to be processed.

[0184] In addition, since the resource configuration of the pending test task and the task parameters of the resource configuration are based on the current state of the resource pool, the "resource configuration of the pending test task and the task parameters of the resource configuration" are more in line with the current state of the resource pool. This can achieve the purpose of dispatching more resources to the pending test task when resources are abundant, and dispatching fewer resources to the pending test task when resources are scarce, which is conducive to improving the use effect of the resource pool (for example, utilization rate, etc.).

[0185] Method Example 2

[0186] In order to better realize the automatic execution of test tasks, the embodiment of the present application also provides an analysis information index database, and the analysis information index database records a large amount of original information (for example, execution description data of completed test tasks) and analysis results obtained by analyzing these original information (for example, the above-mentioned "test scheduling candidate information corresponding to a large number of candidate identifiers", the above-mentioned "first mapping relationship", the above-mentioned "second mapping relationship", the above-mentioned "preset density", etc.), so that when automatically executing a certain test task, the corresponding information can be obtained from the analysis information index database.

[0187] In addition, all analysis results recorded in the above-mentioned "analysis information index database" are obtained by performing data analysis (for example, big data mining analysis, artificial intelligence analysis based on machine learning models, etc.) on a large amount of original information related to the completed test tasks.

[0188] In addition, the embodiment of the present application does not limit the construction method of the above-mentioned "analysis information index database". For example, it can be constructed in an offline manner.

[0189] Method Example 3

[0190] In addition, in order to further improve the automatic testing effect, the above-mentioned "analysis information index database" can be updated in real time. Based on this, the embodiment of the present application also provides another possible implementation of the testing method. In this implementation, in addition to including the above-mentioned S1-S4, the testing method can also include S5:

[0191] S5: After determining that the pending test task is completed, the analysis information index database is updated using the execution description data of the pending test task.

[0192] In an embodiment of the present application, after determining that the automatic execution process for the test task to be processed has been completed, the execution description data of the test task to be processed (for example, whether the test task is completed, what the resource configuration is, the QPS regulated under the resource configuration, the execution time period, the execution duration, the number of repeated executions of the task, etc.) can be used to update the analysis information index database, so that the original information and analysis results recorded in the analysis information index database are updated once, especially all the original information and analysis results related to the test task to be processed in the analysis information index database (for example, the above-mentioned "test scheduling reference information corresponding to the information index identifier", the above-mentioned "first mapping relationship", the above-mentioned "second mapping relationship", the above-mentioned "preset density", etc.) are updated once.

[0193] Based on the relevant content of S5 above, it can be known that after determining that the automatic execution process for the test task to be processed has been completed, the execution description data of the test task to be processed can be used to update the analysis information index database so that the analysis information index database can record the test task analysis information contributed by the test task to be processed, thereby improving the real-time performance of the analysis information index database as much as possible.

[0194] In fact, when analyzing the relevant information of each service under test (for example, the low inflection point of the number of repeated executions of the test task, the low inflection point of the resource demand of the service under test, and the maximum query rate per second QPS corresponding to the low inflection point of the resource demand of the service under test, etc.), it is usually necessary to complete a large number of test tasks for the service under test in advance.

[0195] However, in some cases, the following phenomenon may occur: only a small number of test tasks for a certain service under test are completed, resulting in only a small amount of sample data for the service under test (for example, a small number of execution description data of the test tasks, etc.). Therefore, in order to improve the analysis effect for the service under test, the embodiment of the present application also provides another possible implementation method. In this implementation method, in addition to the above S1-S5, the testing method also includes S6:

[0196] S6: When it is determined that the number of execution description data corresponding to the above-mentioned "test objects of the test tasks to be processed" in the analysis information index database is lower than the second threshold, and when it is determined that the resource pool meets the preset low pressure condition, continue to execute S1 and its subsequent steps.

[0197] The execution description data corresponding to the above-mentioned “test object of the test task to be processed” refers to the execution description data related to the “test object of the test task to be processed” (that is, the execution description data obtained by performing service testing on the test object).

[0198] The preset low-pressure condition can be set in advance. For example, when the data representing the degree of resource tension is larger, it means that the possibility of the current idle resources in the resource pool meeting the resource scheduling requirements of the test task under the estimated task density is smaller, thereby indicating that the resources are more tense. The preset low-pressure condition can be: the data representing the degree of resource tension is lower than the third threshold.

[0199] Based on the relevant content of S6 above, it can be known that if it is determined that the number of execution description data corresponding to the above-mentioned "test objects of test tasks to be processed" in the analysis information index database is lower than the second threshold, it can be determined that there are relatively few service test processes for the test object. Therefore, in order to improve the analysis effect of the test object, S1-S5 can be automatically executed when the resource pool pressure is relatively small. Until it is determined that the number of execution description data corresponding to the test object in the analysis information index database is not lower than the second threshold, the loop execution process of S1-S5 is terminated, and the execution description data corresponding to the test object is analyzed to obtain the analysis result of the test object (for example, test scheduling reference information). In this way, the accuracy of the analysis result can be improved as much as possible without affecting the normal operation of the resource pool.

[0200] Method Example 4

[0201] In order to better understand the test method provided in the embodiment of the present application, Figure 2 The system shown in FIG. Figure 2 A schematic diagram of the engineering architecture of a test system provided in an embodiment of the present application.

[0202] For Figure 2 For the test system shown, the test system is suitable for executing any possible implementation of the test method provided in the embodiment of the present application, so that the test system can automatically execute a certain test task.

[0203] like Figure 2 As shown, the test system provided in the embodiment of the present application can be divided into an online system and an offline system. For ease of understanding, the online system and the offline system are introduced separately below.

[0204] Online System

[0205] for Figure 2 For the online system shown in the figure, the online system refers to the system that acts when there is a task request (for example, the "automatic test request" mentioned above), and the online system can be divided into the following seven modules:

[0206] Online task management module: It is the entrance to automated testing and is responsible for managing user tasks and preprocessing.

[0207] Task engine service: manages the task execution process, initiates sub-module requests, feedback results, node connection, etc.

[0208] Data collection module: responsible for collecting task data from the task engine service, including execution results, execution time, etc.

[0209] Resource scheduling module: responsible for generating the optimal allocation strategy based on the real-time resource pool status and resource optimization services, and allocating resources for test tasks.

[0210] Resource optimization module: Responsible for performing online estimation based on the index data produced by offline analysis, including resource inflection point values, estimated available resource status, and task density.

[0211] Container scheduling module: creates a test environment based on allocated resources.

[0212] Test execution module: responsible for sending test traffic and collecting test results to the created test environment.

[0213] In addition, to further understand Figure 2 The online system shown below uses the automatic execution process of the test tasks to be processed as an example to introduce the working principle of the online system.

[0214] As an example, the working principle of the online system is as follows: after the online task management module receives the automatic test request triggered by the user for the pending test task, the task engine service module initiates a request to the resource scheduling module so that the resource scheduling module can refer to the real-time status of the resource pool obtained by the resource pool management module and the test scheduling reference information obtained by the resource optimization module to allocate resources, obtain the resource configuration of the pending test task, and regulate the maximum acceptable QPS and the number of repeated executions of the task under the resource configuration to obtain the task parameters of the resource configuration; then, the task engine service module deploys the test environment according to the resource configuration, and initiates a request to the test execution module so that the test execution module can execute the pending test task according to the task parameters of the resource configuration, so that after the task engine service module determines that the execution of the pending test task is completed, the task engine service module initiates a request to the data acquisition module so that the data acquisition module can collect task sample data (for example, the execution description data of the pending test task) from the engine service module.

[0215] Based on the relevant content of the above online system, it can be seen that during the peak period of test tasks, the resource status is relatively tight, and a large number of test tasks may be queued for a long time or even fail to deploy due to insufficient resources. Therefore, the resource scheduling module should ensure that as many test tasks as possible are executed correctly. Therefore, within the resource security threshold of the tested service, the resource scheduling module can reduce the resource configuration and the corresponding QPS to ensure that the maximum number of environments is provided, thereby ensuring the concurrency of tasks. However, during the off-peak period of test tasks, there will be a large amount of idle resources in the resource pool, so that the resource scheduling module can utilize the idle resources as much as possible. Therefore, the resource scheduling module can improve the resource configuration of the tested service and the corresponding QPS to optimize the test task time and optimize the user experience. In this way, the function of intelligently selecting resource strategies for the tested service based on the current available resources can be realized.

[0216] Offline system

[0217] for Figure 2 For the offline system shown in the figure, the offline system refers to a module that does not rely on task requests and can perform independent analysis and output. The offline system can be divided into the following three modules:

[0218] Offline task scheduling module: It is the entrance to offline system testing and is responsible for supplementing new task samples based on the data collection sample range.

[0219] Offline data analysis module: responsible for collecting samples based on data and performing data analysis, including sample classification, data fitting, etc.

[0220] Offline index storage module: used to store analysis indexes of offline data.

[0221] In addition, the working principle of the offline system is as follows: the offline task scheduling module can perform statistical analysis on the sample coverage of the large amount of data stored in the data acquisition module, so that when it is determined that the number of a certain type of samples (for example, the number of test tasks used to test XXX software) is relatively small, the offline task scheduling module can initiate a sample supplement execution process (for example, automatically initiate the execution process of the test tasks used to test XXX software) during the off-peak period of task testing to ensure the balance of different types of samples; the offline data analysis module is responsible for performing information analysis and processing on the collected samples (for example, classification statistics, data analysis, feature fitting, etc.), obtaining analysis results, and placing the analysis results in the storage module for use in the next online request.

[0222] Based on the relevant content of the above offline system, it can be seen that the offline data analysis module in the offline system can obtain the minimum resource configuration and optimal resource configuration of the tested service based on some existing data analysis related to the test task (or, analyze the resource configuration used by the tested service under different levels of resource tension, etc.), as well as the maximum QPS that can be tolerated under different resource configurations, so that these analysis results can be used in the subsequent automatic testing process for the tested service.

[0223] Based on the above information about the test system, it can be seen that the test system can intelligently select the appropriate resource configuration and task parameters for each test task. For example, when the test task volume is large, the test system can automatically select the lowest resource configuration for the tested service, reducing the resource usage of each test and allowing the system to support more test tasks. Furthermore, when the test task volume is small, the test system can automatically select the optimal resource configuration for the tested service, thereby improving the QPS of test requests and shortening test time.

[0224] Based on the testing method provided in the above method embodiment, the embodiment of the present application also provides a testing device, which is explained and illustrated below in conjunction with the accompanying drawings.

[0225] Device embodiment

[0226] For technical details of the testing device provided in the device embodiment, please refer to the above method embodiment.

[0227] See also Figure 3 , which is a structural schematic diagram of a testing device provided in an embodiment of the present application.

[0228] The test device 300 provided in the embodiment of the present application includes:

[0229] The receiving unit 301 is configured to receive an automatic test request triggered by a user for a pending test task; wherein the automatic test request carries an information index identifier of a test object of the pending test task;

[0230] A query unit 302 is configured to query a pre-built analysis information index database for test scheduling reference information corresponding to the information index identifier; wherein the test scheduling reference information is used to indicate the minimum conditions required to be met when scheduling resources for the pending test task;

[0231] The determining unit 303 is configured to determine the resource configuration of the to-be-processed test task and the task parameters of the resource configuration according to the current state of the resource pool and the test scheduling reference information;

[0232] The testing unit 304 is configured to execute the pending test task according to the resource configuration and the task parameters of the resource configuration.

[0233] In a possible implementation, the test scheduling reference information includes at least one of a low inflection point of the number of test task repetitions, a low inflection point of the tested service resource demand, and a maximum query rate per second (QPS) corresponding to the low inflection point of the tested service resource demand.

[0234] In a possible implementation, the test scheduling reference information includes a low inflection point of the number of test task repetitions, a low inflection point of the resource demand of the tested service, and a maximum QPS value corresponding to the low inflection point of the resource demand of the tested service;

[0235] The determining unit 303 includes:

[0236] an allocation subunit, configured to perform resource allocation processing on the pending test task according to the current state of the resource pool and the low inflection point of the resource demand of the tested service, so as to obtain the resource configuration of the pending test task;

[0237] The control subunit is used to determine the task parameters of the resource configuration according to the low inflection point of the number of repeated executions of the test task and the QPS maximum value corresponding to the low inflection point of the resource demand of the tested service.

[0238] In a possible implementation, the allocation subunit includes:

[0239] A first determining subunit, configured to determine current idle resources in the resource pool according to a current state of the resource pool;

[0240] The second determining subunit is configured to perform resource allocation processing on the pending test task according to the current idle resources and the low inflection point of resource demand of the tested service, so as to obtain resource configuration of the pending test task.

[0241] In a possible implementation, the testing device 300 further includes:

[0242] An acquisition unit, configured to acquire an estimated task density; wherein the estimated task density is used to represent the number of test tasks predicted to be executed simultaneously during the execution of the to-be-processed test task;

[0243] The second determining subunit is specifically used to: perform resource allocation processing on the pending test task according to the estimated task density, the current idle resources, and the low inflection point of the resource demand of the tested service, to obtain the resource configuration of the pending test task.

[0244] In one possible implementation, the second determination sub-unit is specifically used to: determine resource tension characterization data based on the estimated task density and the resource amount of the current idle resources; if the resource tension characterization data meets the preset resource tension condition, then according to the low inflection point of the resource demand of the tested service, perform resource allocation processing on the test task to be processed from the current idle resources to obtain the resource configuration of the test task to be processed.

[0245] In a possible embodiment, the second determination sub-unit is also used to: if the resource tension characterization data does not meet the preset resource tension condition, query the first adjustment information corresponding to the resource tension characterization data at the low inflection point of the resource demand of the tested service from a pre-constructed first mapping relationship; use the first adjustment information and the low inflection point of the resource demand of the tested service to determine the amount of resources to be used; according to the amount of resources to be used, perform resource allocation processing on the test task to be processed from the current idle resources to obtain the resource configuration of the test task to be processed.

[0246] In a possible implementation, the estimated task density is obtained by prediction using execution description data of test tasks executed within a historical time period.

[0247] In one possible implementation, the control sub-unit is specifically used to: if the resource configuration of the test task to be processed meets the preset minimum configuration conditions, then the low inflection point of the number of repeated executions of the test task and the maximum QPS value corresponding to the low inflection point of the resource demand of the service under test are determined as the task parameters of the resource configuration.

[0248] In a possible embodiment, the control sub-unit is also used to: if the resource configuration of the test task to be processed meets the preset adjustment condition, then compare the resource configuration of the test task to be processed with the low inflection point of the resource demand of the service under test to obtain a resource abundance characterization value; query the second adjustment information corresponding to the resource abundance characterization value under the low inflection point of the number of repeated executions of the test task and the QPS maximum value corresponding to the low inflection point of the resource demand of the service under test from a pre-constructed second mapping relationship; use the second adjustment information, the low inflection point of the number of repeated executions of the test task, and the QPS maximum value corresponding to the low inflection point of the resource demand of the service under test to determine the task parameters of the resource configuration.

[0249] In a possible implementation, the testing device 300 further includes:

[0250] An updating unit is configured to update the analysis information index database using the execution description data of the test task to be processed after determining that the test task to be processed is completed.

[0251] Based on the relevant content of the above-mentioned test device 300, it can be known that for the test device 300 provided in the embodiment of the present application, after receiving the automatic test request triggered by the user for the pending test task, the information index identifier of the test object of the pending test task carried by the automatic test request is first used to query the test scheduling reference information corresponding to the information index identifier from the pre-built analysis information index database, so that the test scheduling reference information can represent the minimum conditions required to be achieved when performing resource scheduling for the pending test task; then, based on the test scheduling reference information and the current status of the resource pool, the resource configuration of the pending test task and the task parameters of the resource configuration are determined; finally, the pending test task is executed according to the resource configuration and the task parameters of the resource configuration, so that the purpose of automatically executing the pending test task can be achieved, thereby effectively avoiding the adverse effects caused by manually driven test tasks (for example, the need to consume a large amount of resources), and thus effectively reducing the resource consumption of the testing process.

[0252] In addition, since the test scheduling reference information is used to indicate the minimum conditions that need to be met when scheduling resources for the test tasks to be processed (especially, the minimum amount of resources required to be used), the resource configuration and task parameters determined based on the test scheduling reference information can at least meet the minimum conditions, so that the resource configuration and task parameters can ensure the smooth execution of the test tasks to be processed. This can effectively avoid the defect that the test tasks to be processed cannot be executed or fail to execute due to insufficient resources scheduled for the test tasks to be processed, thereby improving the automatic execution effect of the test tasks to be processed.

[0253] In addition, since the resource configuration of the pending test task and the task parameters of the resource configuration are based on the current state of the resource pool, the "resource configuration of the pending test task and the task parameters of the resource configuration" are more in line with the current state of the resource pool. This can achieve the purpose of dispatching more resources to the pending test task when resources are abundant, and dispatching fewer resources to the pending test task when resources are scarce, which is conducive to improving the use effect of the resource pool (for example, utilization rate, etc.).

[0254] Furthermore, an embodiment of the present application also provides a device, comprising a processor and a memory:

[0255] The memory is used to store computer programs;

[0256] The processor is used to execute any implementation of the testing method provided in the embodiments of the present application according to the computer program.

[0257] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute any implementation of the test method provided in the embodiment of the present application.

[0258] Furthermore, an embodiment of the present application also provides a computer program product, which, when running on a terminal device, enables the terminal device to execute any implementation of the testing method provided in the embodiment of the present application.

[0259] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0260] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as above with a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present invention using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A testing method, characterized in that: The method comprises: Receive an automatic test request triggered by a user for a pending test task; wherein the automatic test request carries an information index identifier of a test object of the pending test task; Querying the test scheduling reference information corresponding to the information index identifier from a pre-built analysis information index database; wherein the test scheduling reference information is used to indicate the minimum conditions required to be met when scheduling resources for the pending test task, and the test scheduling reference information includes a low inflection point of the number of repeated executions of the test task, a low inflection point of the resource demand of the tested service, and a maximum query rate per second (QPS) corresponding to the low inflection point of the resource demand of the tested service; Determine the resource configuration of the pending test task and the task parameters of the resource configuration according to the current state of the resource pool and the test scheduling reference information, wherein the task parameters are determined according to the low inflection point of the number of repeated executions of the test task and the maximum QPS; the resource configuration is obtained by performing resource allocation processing on the pending test task according to the estimated task density, the current idle resources of the resource pool, and the low inflection point of the resource demand of the tested service; the estimated task density is used to indicate the number of test tasks predicted to be executed simultaneously during the execution of the pending test task; and the current idle resources are determined according to the current state; Execute the pending test task according to the resource configuration and the task parameters of the resource configuration.

2. The method according to claim 1, characterized in that The resource allocation determination process includes: Determining resource stress level characterization data based on the estimated task density and the amount of the currently idle resources; If the resource stress level characterization data meets the preset resource stress condition, resource allocation processing is performed on the pending test task from the current idle resources according to the low inflection point of the resource demand of the tested service to obtain the resource configuration of the pending test task.

3. The method according to claim 2, characterized in that The method further comprises: If the resource stress level characterization data does not meet the preset resource stress condition, querying the first increase information corresponding to the resource stress level characterization data at the low inflection point of the resource demand of the measured service from a pre-constructed first mapping relationship; Determining the amount of resources to be used by using the first increase information and the low inflection point of the resource demand of the measured service; According to the amount of resources to be used, resource allocation processing is performed on the test task to be processed from the current idle resources to obtain a resource configuration of the test task to be processed.

4. The method according to claim 1, wherein The estimated task density is obtained by prediction using execution description data of test tasks executed in a historical time period.

5. The method according to claim 1, wherein The process of determining the task parameters includes: If the resource configuration of the test task to be processed meets the preset minimum configuration conditions, the low inflection point of the number of repeated executions of the test task and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service are determined as the task parameters of the resource configuration.

6. The method according to claim 5, characterized in that The method further comprises: If the resource configuration of the pending test task meets the preset increase condition, the resource configuration of the pending test task is compared with the low inflection point of the resource demand of the tested service to obtain a resource abundance representation value; Querying, from a pre-constructed second mapping relationship, second increase information corresponding to the resource abundance representation value at the maximum QPS value corresponding to the low inflection point of the number of repeated executions of the test task and the low inflection point of the resource demand of the tested service; The task parameters of the resource configuration are determined by using the second adjustment information, the low inflection point of the number of repeated executions of the test task, and the maximum QPS value corresponding to the low inflection point of the resource demand of the tested service.

7. The method according to claim 1, characterized in that The method further comprises: After determining that the pending test task is completed, the analysis information index database is updated using the execution description data of the pending test task.

8. A testing device, characterized in that: include: A receiving unit, configured to receive an automatic test request triggered by a user for a pending test task; wherein the automatic test request carries an information index identifier of a test object of the pending test task; A query unit, configured to query the test scheduling reference information corresponding to the information index identifier from a pre-constructed analysis information index database; wherein the test scheduling reference information is used to indicate the minimum conditions required to be met when scheduling resources for the test task to be processed, and the test scheduling reference information includes a low inflection point of the number of repeated executions of the test task, a low inflection point of the resource demand of the tested service, and a maximum query rate per second (QPS) corresponding to the low inflection point of the resource demand of the tested service; a determination unit, configured to determine the resource configuration of the pending test task and task parameters of the resource configuration based on the current state of the resource pool and the test scheduling reference information, wherein the task parameters are determined based on a low inflection point of the number of repeated executions of the test task and the maximum QPS; the resource configuration is obtained by performing resource allocation processing on the pending test task based on an estimated task density, current idle resources of the resource pool, and a low inflection point of resource demand for the tested service; the estimated task density is used to indicate the number of test tasks predicted to be executed simultaneously during the execution of the pending test task; and the current idle resources are determined based on the current state; The testing unit is used to execute the pending test task according to the resource configuration and the task parameters of the resource configuration.

9. A device, characterized in that The device includes a processor and a memory: The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 7 according to the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that When the computer program product is run on a terminal device, the terminal device is enabled to execute the method according to any one of claims 1 to 7.

Images