Performance test method and device for equipment

Abstract

The invention discloses a device performance test method and device, which can be applied to various scenes such as cloud technology, artificial intelligence, smart traffic, Internet of Vehicles and the like, and the method comprises the following steps: obtaining a preset qps upper limit threshold and a preset qps lower limit threshold of a device; testing the equipment based on a preset qps lower limit threshold to obtain an initial test result; if the initial timeout rate is greater than or equal to a preset timeout rate threshold value, iterating the qps lower limit threshold value by adopting a dichotomy method to obtain a next round of qps threshold value; if the initial timeout rate is greater than or equal to a task conclusion timeout rate threshold value and less than a preset timeout rate threshold value, determining a next round of qps threshold value based on a qps lower limit threshold value and unit iteration qps; if the initial timeout rate is smaller than a task conclusion timeout rate threshold value, determining a next round of qps threshold value based on a preset qps upper limit threshold value; and testing the equipment based on the next round of qps threshold to obtain a performance test result. The performance test result of the equipment can be quickly determined, and the test efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and apparatus for testing the performance of a device. Background Technology

[0002] The performance test conclusion for backend service devices is generally the maximum QPS supported by a single machine. This QPS is determined by constraining the load test requests under a certain timeout rate. As the QPS increases, the CPU load on the backend service device increases, the processing time of the load test requests increases, and thus the timeout rate also increases. The goal of performance testing is to find the maximum QPS of a single machine under a certain timeout rate constraint.

[0003] In the process of device performance testing in related technologies, a small initial QPS is usually set, then an appropriate QPS iteration step size is set, and then it is iterated from small to large until the timeout rate exceeds the threshold. The QPS at this point is then used as the performance conclusion. This testing method involves a large number of iterations, resulting in low performance testing efficiency. Summary of the Invention

[0004] This application provides a method and apparatus for testing the performance of a device, which can reduce the number of iterations in the performance testing process and quickly determine the performance test results of the device, thereby improving testing efficiency.

[0005] On the one hand, this application provides a method for testing the performance of a device, the method comprising:

[0006] Obtain the device's preset upper limit threshold and preset lower limit threshold for QPS;

[0007] The device is tested based on the preset lower limit threshold qps to obtain initial test results; the initial test results include the initial timeout rate.

[0008] If the initial timeout rate is greater than or equal to the preset timeout rate threshold, the QPS lower limit threshold is iterated using a binary search method to obtain the QPS threshold for the next round; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than a first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test process;

[0009] If the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold, the next round of QPS threshold is determined based on the QPS lower limit threshold and the unit iteration QPS.

[0010] If the initial timeout rate is less than the task conclusion timeout rate threshold, the next round of qps threshold is determined based on the preset qps upper limit threshold.

[0011] The device is tested based on the next round of QPS threshold to obtain performance test results.

[0012] On the other hand, a performance testing apparatus for a device is provided, the apparatus comprising:

[0013] The limit threshold acquisition module is used to acquire the preset upper limit threshold and the preset lower limit threshold of the device's QPS.

[0014] The initial result determination module is used to test the device based on the preset lower limit threshold of qps and obtain initial test results; the initial test results include the initial timeout rate;

[0015] The first threshold determination module is used to obtain the next round of QPS threshold by iterating the lower limit threshold of QPS using a binary search method if the initial timeout rate is greater than or equal to a preset timeout rate threshold; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than the first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test process;

[0016] The second threshold determination module is used to determine the next round qps threshold based on the qps lower limit threshold and the unit iteration qps if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold.

[0017] The third threshold determination module is used to determine the next round qps threshold based on the preset qps upper limit threshold if the initial timeout rate is less than the task conclusion timeout rate threshold.

[0018] The test result determination module is used to test the device based on the next round of qps threshold and obtain performance test results.

[0019] On the other hand, an electronic device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the performance testing method of the device as described above.

[0020] On the other hand, a computer storage medium is provided that stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the performance testing method of the device as described above.

[0021] On the other hand, a computer program product is provided, including a computer program that is loaded and executed by a processor to implement the performance testing method of the device as described above.

[0022] The performance testing method and apparatus for the equipment provided in this application have the following technical advantages:

[0023] This application obtains a preset upper limit threshold and a preset lower limit threshold for the server; tests the server based on the preset lower limit threshold to obtain initial test results; the initial test results include an initial timeout rate; if the initial timeout rate is greater than or equal to a preset timeout rate threshold, a binary search method is used to iterate the lower limit threshold to obtain the next round of QPS threshold; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than a first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test; if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold, the next round of QPS threshold is determined based on the lower limit threshold and the unit iteration QPS; if the initial timeout rate is less than the task conclusion timeout rate threshold, the next round of QPS threshold is determined based on the preset upper limit threshold; thus, the next round of QPS threshold can be quickly and accurately determined under different conditions, and the server can be tested based on the next round of QPS threshold to obtain performance test results. This application enables the setting of different QPS thresholds for the next round based on different situations, reducing the number of iterations in the performance testing process and enabling the rapid determination of device performance test results, thereby improving testing efficiency. Attached Figure Description

[0024] To more clearly illustrate the technical solutions and advantages in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the application environment of a device performance testing method provided in the embodiments of this specification;

[0026] Figure 2 This is a schematic flowchart of a device performance testing method provided in the embodiments of this specification;

[0027] Figure 3 This is a flowchart illustrating a method for predicting the third QPS of upward iteration and the fourth QPS of downward iteration, as provided in the embodiments of this specification.

[0028] Figure 4This is a flowchart illustrating a method provided in the embodiments of this specification for sorting test result set data according to QPS and predicting the third QPS for upward iteration and the fourth QPS for downward iteration.

[0029] Figure 5 This is a flowchart illustrating a method for determining the third QPS for upward iteration based on the correlation between CPU load and QPS, as provided in the embodiments of this specification.

[0030] Figure 6 This is a flowchart illustrating a method for determining the third QPS for upward iteration based on correlation and target CPU load, as provided in an embodiment of this specification.

[0031] Figure 7 This is a flowchart illustrating a method for iterative testing based on the current test result set of the current round, as provided in the embodiments of this specification.

[0032] Figure 8 This is a flowchart illustrating a method for determining the target upper limit of QPS for a device, as provided in an embodiment of this specification.

[0033] Figure 9-11 This is a schematic diagram of a test interface for a server performance testing platform provided in the embodiments of this specification;

[0034] Figure 12 This is a flowchart illustrating another device performance testing method provided in the embodiments of this specification;

[0035] Figure 13 This is a schematic diagram of the structure of a performance testing device for an equipment provided in the embodiments of this specification;

[0036] Figure 14 This is a schematic diagram of the structure of a server provided in the embodiments of this specification. Detailed Implementation

[0037] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0039] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0040] Please see Figure 1 , Figure 1 This is a schematic diagram of the application environment of a device performance testing method provided in an embodiment of this application. The application environment may include at least a server 100 and a terminal 200.

[0041] In an optional embodiment, server 100 can be used to perform performance testing on the device. Server 100 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0042] In an optional embodiment, terminal 200 can be used to display the performance test results of the device. Specifically, terminal 200 can be, but is not limited to, electronic devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, in-vehicle terminals, and smart TVs; it can also be software running on the aforementioned electronic devices, such as applications or mini-programs. The operating system running on the electronic device in this embodiment can include, but is not limited to, Android, iOS, Linux, and Windows.

[0043] The parameters involved in the testing process in this embodiment are as follows:

[0044] min_qps: QPS in the first round of testing. QPS (queries-per-second) is a measure of how much traffic a specific query server processes within a specified time.

[0045] max_qps: The QPS for the second round of testing. The QPS value for the second round of testing may be adjusted based on the QPS test results of the first round. Therefore, max_qps here is just an upper bound. The second round of load testing will only be performed when the QPS test results of the first round meet certain conditions.

[0046] perf_data_list: Used to record the results of each round of QPS test, mainly including QPS, CPU load, and timeout rate;

[0047] replay_dict_info: Used to record information about replaying tests at the same QPS; replaying a test means repeating the test at the same QPS.

[0048] qps_step: The smallest unit in the qps iteration, also used to determine the stopping condition;

[0049] task_conclusion_cpu_threshold: An empirical threshold for CPU utilization, representing the CPU utilization at which the test conclusion reaches the maximum QPS under normal circumstances. For example, this value can be set to 80%.

[0050] task_conclusion_error_rate: The task conclusion timeout rate threshold. The test conclusion is the maximum QPS under this timeout rate constraint. For example, this value can be set to 0.1%.

[0051] `task_end_error_rate`: This is the timeout rate threshold for task termination. Only when the timeout rate for a given round of QPS exceeds this threshold will the QPS stop increasing. The purpose of this value is to mitigate fluctuations during testing. For example, if the timeout rate for a given round of QPS is 0.12%, and this timeout rate is very close to `task_conclusion_error_rate`, to prevent this 0.12% from being a fluctuation, it is necessary to continue iterating upwards in QPS to ensure the timeout rate is greater than or equal to `task_end_error_rate`. `task_end_error_rate` is generally set to be slightly greater than `task_conclusion_error_rate`, such as 0.3%.

[0052] lower_bound_binary_search_error_rate: Preset timeout rate threshold, which is the QPS timeout rate threshold when using the binary search method to iterate down the QPS. It is generally set to be much higher than task_conclusion_error_rate, for example, set to 10%. The reason for setting this parameter is that it is not always necessary to use the binary search method to iterate down the QPS. For example, if the current QPS timeout rate is already very close to task_conclusion_error_rate, the QPS can be reduced by fine-tuning in order to obtain test results more quickly.

[0053] Stopping condition: Stop when the difference between the minimum QPS above task_conclusion_error_rate and the maximum QPS below task_conclusion_error_rate is less than or equal to qps_step;

[0054] Fluctuation: When the timeout rate of high QPS is less than task_conclusion_error_rate, while the timeout rate of low QPS is greater than task_conclusion_error_rate, fluctuation is considered to have occurred.

[0055] Rerun mechanism: Calculate the absolute value of the gap between the timeout rate and task_conclusion_error_rate for high QPS and low QPS respectively, and prioritize rerunning QPS with smaller absolute gap values.

[0056] The following describes a performance testing method for the device according to this application. Figure 2 This is a flowchart illustrating a device performance testing method provided in an embodiment of this specification. This specification provides the operational steps of the method described in the embodiment or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiment is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiment or drawings. Specifically, as... Figure 2 As shown, the above method may include:

[0057] S201: Obtain the device's preset QPS upper limit threshold and preset QPS lower limit threshold.

[0058] In the embodiments of this specification, the tested device may include, but is not limited to, a backend server. The aforementioned acquisition of the preset upper limit threshold and preset lower limit threshold of the device may include:

[0059] Obtain the historical QPS dataset obtained from testing the above devices within a historical time period;

[0060] The minimum value in the above historical qps dataset is determined as the above qps lower limit threshold.

[0061] The maximum value in the historical QPS dataset is determined as the QPS upper limit threshold.

[0062] In this embodiment of the disclosure, the historical time period can be the time period corresponding to the target number of tests closest to the current time. The target number can be set according to the actual situation and is not specifically limited here. For example, the target number can be 5, and the results of the 5 most recent historical tests of the backend service under test can be obtained. The minimum QPS is set as the lower limit threshold (min_qps), and the maximum QPS is set as the upper limit threshold (max_qps). In some embodiments, if there are no historical test results, the pre-set default min_qps and max_qps are used.

[0063] S203: Test the above-mentioned device based on the above-mentioned preset lower limit threshold of qps to obtain the initial test results; the above-mentioned initial test results include the initial timeout rate.

[0064] In the embodiments described in this specification, the initial test results may also include the initial CPU load. During the test, the test is first performed starting from a preset lower limit threshold for QPS, and then the test is iterated upwards or downwards using a binary search method.

[0065] S205: If the initial timeout rate is greater than or equal to the preset timeout rate threshold, the lower limit of QPS is iterated using a binary search method to obtain the QPS threshold for the next round; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than the first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test.

[0066] In the embodiments of this specification, the aforementioned preset timeout rate threshold (lower_bound_binary_search_error_rate) is the threshold used for downward iteration using the aforementioned binary search method. The preset timeout rate threshold can be determined based on the task conclusion timeout rate threshold, and the difference between the aforementioned preset timeout rate threshold and the aforementioned task conclusion timeout rate threshold is greater than a first threshold, which can be determined based on actual conditions. The test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test process. When the initial timeout rate is greater than or equal to lower_bound_binary_search_error_rate, the QPS is directly iterated downward using the binary search method, i.e., the QPS for the next round is set to min_qps / 2, that is, half of the lower limit threshold of QPS is set as the QPS threshold for the next round.

[0067] S207: If the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold, the next round of QPS threshold is determined based on the lower limit threshold of QPS and the QPS per unit iteration.

[0068] In the embodiments of this specification, if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold, the next round of QPS threshold can be determined based on the QPS lower limit threshold and the QPS per unit iteration.

[0069] S209: If the initial timeout rate is less than the task conclusion timeout rate threshold, the next round of QPS threshold is determined based on the preset QPS upper limit threshold.

[0070] In the embodiments of this specification, if the initial timeout rate is less than task_conclusion_error_rate, the preset qps upper limit threshold is used to determine the qps threshold of the next round, and the qps of the next round is max_qps, which is used to initially anchor the upper limit of performance.

[0071] S2011: Test the above devices based on the next round of qps threshold to obtain performance test results.

[0072] In the embodiments of this specification, different QPS thresholds for the next round can be determined according to different initial test results, and the device can be tested in the next round to obtain performance test results, thereby improving test efficiency.

[0073] In the embodiments of this specification, if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold, determining the next round QPS threshold based on the lower limit QPS threshold and the unit iteration QPS includes:

[0074] If the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the task stop timeout rate threshold, the next round of QPS threshold is determined by iterating upward based on the lower limit threshold of QPS and the unit iteration QPS, where it refers to increasing QPS; the difference between the task stop timeout rate threshold and the task conclusion timeout rate threshold is less than the second threshold, and the task stop timeout rate threshold is less than the preset timeout rate threshold.

[0075] If the initial timeout rate is greater than or equal to the task stop timeout rate threshold, but less than the preset timeout rate threshold, the next round of QPS threshold is determined based on the lower limit threshold of QPS and the unit iteration QPS. The downward iteration refers to reducing QPS.

[0076] In the embodiments of this specification, the above-mentioned task stop timeout rate threshold is determined based on the above-mentioned task conclusion timeout rate threshold and is greater than the above-mentioned task conclusion timeout rate threshold, and the above-mentioned second threshold is less than the above-mentioned first threshold.

[0077] In the embodiments of this specification, if the initial timeout rate is greater than or equal to task_end_error_rate and less than owner_bound_binary_search_error_rate, then the load test is performed by reducing the qps_step (unit iteration qps) by n times. That is, the qps of the next round is min_qps-n*qps_step, where n is a multiple of 2. For example, n can be set to 4. If min_qps-4*step is less than 0, then the qps of the next round is set to min_qps-qps_step. It is necessary to ensure that the qps of the next round is at least greater than 0.

[0078] In the embodiments of this specification, if the initial timeout rate is greater than or equal to task_conclusion_error_rate and less than task_end_error_rate, then the QPS of the next round is min_qps + qps_step, that is, iterates upwards one round to ensure that the current timeout rate is not a fluctuating result.

[0079] In this embodiment of the specification, the device is tested based on the aforementioned next-round QPS threshold to obtain performance test results, including:

[0080] Based on the above-mentioned next round QPS threshold, the above-mentioned devices are tested to obtain a first timeout rate and a second timeout rate. The first timeout rate is greater than the above-mentioned task conclusion timeout rate threshold; the second timeout rate is less than or equal to the above-mentioned task conclusion timeout rate threshold.

[0081] The minimum QPS during the test process is obtained by obtaining the first QPS;

[0082] The maximum QPS of the second timeout rate during the test process is obtained to get the second QPS;

[0083] Calculate the difference between the first QPS and the second QPS to obtain the target difference;

[0084] If the target difference is less than or equal to the unit iteration QPS, the test will be stopped and the test results will be obtained.

[0085] If the target difference is greater than the unit iteration qps, continue to execute the next round of testing to obtain the performance test results.

[0086] In the embodiments of this specification, iterative testing refers to testing the device using an initially set QPS threshold, then re-determining the QPS threshold for the next round of testing based on the initial test results and the initially set QPS threshold, and continuing to test the device in the next round based on the re-determined QPS threshold to obtain the next round of test results; then re-determining the QPS threshold based on the next round of test results and the next round of test results to continue testing the device, and repeating this cycle until the test ends. The test stops when the difference between the minimum QPS above `task_conclusion_error_rate` and the maximum QPS below `task_conclusion_error_rate` is less than or equal to `qps_step`. During iterative testing, the minimum QPS at the first timeout rate is obtained, resulting in the first QPS; the maximum QPS at the second timeout rate is obtained, resulting in the second QPS. The difference between the first and second QPS is then calculated to obtain the target difference. The target difference is then compared to the unit iteration QPS. If the target difference is less than or equal to the unit iteration QPS, the iterative test stops, meaning it stops further iterations, and the current QPS is taken as the test result. If the target difference is greater than the unit iteration QPS, the next round of testing can continue until the performance test result is obtained. This allows for determining whether to execute iterative tests based on the unit iteration QPS, improving iterative testing efficiency.

[0087] In the embodiments described in this specification, such as Figure 3 As shown, the above method also includes:

[0088] S301: If the number of test rounds exceeds a preset threshold, determine whether there are any rounds that meet the rerun mechanism.

[0089] S303: If there are rounds that satisfy the rerun mechanism, determine the target QPS corresponding to the rerun mechanism, and perform the rerun test based on the target QPS.

[0090] S305: If there is no round that satisfies the rerun mechanism, obtain the test result set of each round of testing of the above equipment, sort the data in the above test result set according to QPS, and predict the third QPS for upward iteration and the fourth QPS for downward iteration.

[0091] In the embodiments of this specification, the preset threshold can be set according to the time situation. For example, the preset threshold can be set to 2 times. When the number of test rounds is greater than the preset threshold, it can be determined whether there is a round that meets the rerun mechanism based on the third timeout rate, the fourth timeout rate, and the task conclusion timeout rate threshold. The third QPS is the highest QPS in the test process; the fourth QPS is the lowest QPS in the test process. For the rounds that need to be rerun, the target QPS corresponding to the rerun mechanism can be determined, and the rerun test can be performed based on the above target QPS. If there is no round that needs to be rerun, the test result set (perf_data_list) of the current tested rounds can be obtained. The test result set can be used to record the results of each round of QPS test, mainly including QPS, CPU load, and timeout rate. The data in the test result set is sorted according to QPS, and the third QPS for upward iteration and the fourth QPS for downward iteration are predicted, so as to perform subsequent iterative tests, thereby reducing the number of test iterations and quickly obtaining the maximum QPS under a single server machine.

[0092] In the embodiments of this specification, the above determination of whether there is a round that satisfies the rerun mechanism includes:

[0093] The above devices were tested based on the third QPS, and the third timeout rate was obtained;

[0094] The fourth timeout rate was obtained by testing the above devices based on the fourth QPS.

[0095] If the third timeout rate is less than the task conclusion timeout rate threshold and the fourth timeout rate is greater than the task conclusion timeout rate threshold, it is determined that there is a round that satisfies the rerun mechanism.

[0096] If the third timeout rate is greater than or equal to the task conclusion timeout rate threshold, and / or the fourth timeout rate is less than or equal to the task conclusion timeout rate threshold, it is determined that there are no rounds that meet the rerun mechanism.

[0097] In the embodiments of this specification, the third QPS is the highest QPS during the test process; the fourth QPS is the lowest QPS during the test process. When the timeout rate of the high QPS is less than the task_conclusion_error_rate, while the timeout rate of the low QPS is greater than the task_conclusion_error_rate, it is considered that a fluctuation has occurred. If the third timeout rate is less than the task conclusion timeout rate threshold and the fourth timeout rate is greater than the task conclusion timeout rate threshold, it is determined that the test process has fluctuated and there are rounds that meet the rerun mechanism. If the third timeout rate is greater than or equal to the task conclusion timeout rate threshold, and / or the fourth timeout rate is less than or equal to the task conclusion timeout rate threshold, it is determined that the test process has not fluctuated, that is, there are no rounds that meet the rerun mechanism. By comparing the timeout rates of the high QPS and low QPS with the task conclusion timeout rate threshold respectively, the rounds that meet the rerun mechanism are determined, and the rerun test is performed, thereby further improving the test accuracy.

[0098] In the embodiments of this specification, the target QPS corresponding to the rerun mechanism mentioned above includes:

[0099] Calculate the absolute value of the first difference between the third timeout rate and the task conclusion timeout rate threshold, and calculate the absolute value of the second difference between the fourth timeout rate and the task conclusion timeout rate threshold.

[0100] If the absolute value of the first difference is less than or equal to the absolute value of the second difference, the absolute value of the first difference is determined as the target QPS.

[0101] If the absolute value of the first difference is greater than the absolute value of the second difference, the absolute value of the second difference is determined as the target QPS.

[0102] In the embodiments of this specification, after determining the absolute value of the first difference between the third timeout rate and the task conclusion timeout rate threshold, and the absolute value of the second difference between the fourth timeout rate and the task conclusion timeout rate threshold, the target QPS corresponding to the rerun mechanism can be determined based on the absolute value of the first difference and the absolute value of the second difference. For example, the smaller of the absolute value of the first difference and the absolute value of the second difference can be determined as the target QPS. When the absolute value of the first difference is less than or equal to the absolute value of the second difference, the absolute value of the first difference is determined as the target QPS; when the absolute value of the first difference is greater than the absolute value of the second difference, the absolute value of the second difference is determined as the target QPS.

[0103] In the embodiments described in this specification, the above method further includes:

[0104] When the number of retries is less than or equal to the preset threshold, the highest QPS in the above retries test process is redefined as the third QPS, and the lowest QPS in the above retries test process is redefined as the fourth QPS.

[0105] Jump to the steps described above for testing the device based on the third QPS to obtain the third timeout rate;

[0106] When the number of reruns exceeds a preset threshold, the rerun test is terminated.

[0107] In this embodiment, the preset threshold for the number of retries can be set according to actual conditions. For example, the preset threshold can be set to 2. When the number of retries is less than or equal to the preset threshold, the highest QPS in the rerun test process can be redefined as the third QPS, and the lowest QPS in the rerun test process can be redefined as the fourth QPS. Then, the test jumps to the step of testing the device based on the third QPS to obtain the third timeout rate, and this step and other steps after this step are repeated to execute the rerun mechanism. When the number of retries exceeds the preset threshold, the rerun test is exited. In this embodiment, by setting a preset threshold, the unlimited execution of the rerun mechanism can be avoided, thereby improving test efficiency.

[0108] In the embodiments described in this specification, the qps can be iterated according to the following strategy:

[0109] 1) Are there any rounds that need to be rerun? If not, proceed to step 2); if so, rerun them. After the rerun is completed, reassess whether it is necessary to continue rerunning. If the number of reruns exceeds the threshold, such as 2 times, exit the rerun test; otherwise, proceed to step 2).

[0110] 2) Sort the existing perf_data_list by qps and predict the qps for upward and downward iterations, hereinafter referred to as high_qps and lower_qps. The qps for the current round is referred to as current_qps.

[0111] In the embodiments described in this specification, the above performance test results include CPU load and timeout rate, such as... Figure 4 As shown, the test result set data is sorted by QPS, and the prediction of the third QPS for upward iteration and the fourth QPS for downward iteration are as follows:

[0112] S401: Get the current CPU load and current timeout rate corresponding to the current round's QPS;

[0113] S403: If the current CPU load is greater than or equal to a preset value, and the current timeout rate is greater than or equal to the task conclusion timeout rate threshold, calculate the sum of the current round QPS and the unit iteration QPS to obtain the third QPS for upward iteration;

[0114] S405: Sort the data in the above test result set according to QPS and predict the fourth QPS for the next iteration.

[0115] In the embodiments of this specification, the server performance test results may include, but are not limited to, CPU load, timeout rate, service failure rate, average number of advertisements, GPU load, etc., with the iterative test plan mainly constructed based on CPU load and timeout rate. Preset values ​​can be set according to actual conditions; for example, they can be set to 80%, which allows for the rapid determination of the third QPS for the upward iteration and the fourth QPS for the downward iteration based on the current CPU load and current timeout rate corresponding to the current round of QPS.

[0116] In the embodiments described in this specification, such as Figure 5 As shown, the above method also includes:

[0117] S500: Compares the current CPU load with a preset value;

[0118] S501: If the current CPU load is greater than or equal to the preset value, and the current timeout rate is less than the task conclusion timeout rate threshold, calculate the sum of the current round QPS and n times the unit iteration QPS to obtain the third QPS for upward iteration; n is greater than or equal to 2.

[0119] S503: If the current CPU load is less than the preset value, predict the target CPU load based on the current CPU load and the preset CPU load threshold.

[0120] S505: Based on the CPU load data and QPS data after sorting the above test results, fit the correlation between CPU load and QPS, and determine the third QPS for upward iteration based on the correlation and the target CPU load.

[0121] In the embodiments of this specification, if the CPU load corresponding to the current round's QPS is greater than or equal to 80%, and the timeout rate is greater than or equal to the task conclusion timeout rate threshold (task_conclusion_error_rate), then high_qps is current_qps + qps_step, to ensure that the timeout rate of two consecutive rounds of QPS is greater than task_conclusion_error_rate; if the CPU load corresponding to the current round's QPS is greater than or equal to 80%, and the timeout rate is less than task_conclusion_error_rate, then high_qps = current_qps + 4 * qps_step; if the CPU load corresponding to the current round's QPS is less than 80%, the target CPU load can be predicted based on the current CPU load and the preset CPU load threshold; where n is a multiple of 2; for example, n can be 4.

[0122] In the embodiments described in this specification, such as Figure 6 As shown, the third QPS for upward iteration is determined based on the correlation and the target CPU load, including:

[0123] S5051: Calculate the average of the target CPU load and the target value to obtain the target load threshold;

[0124] S5053: Compare the target load threshold with the preset CPU load threshold, and determine the smaller value between the target load threshold and the preset CPU load threshold as the CPU load to be filtered.

[0125] S5055: Based on the above correlation, determine the QPS corresponding to the above-screened CPU load as the third QPS for upward iteration.

[0126] In the embodiments of this specification, the target value can be set according to the actual situation, for example, it can be set to 1. The calculation formula for the target load threshold can be: (1+current_cpu_workload) / 2; the preset CPU load threshold of the server can be task_conclusion_cpu_threshold; compare the above target load threshold with the preset CPU load threshold, and determine the smaller value between the above target load threshold and the above preset CPU load threshold as the filtered CPU load; for example, if the CPU load corresponding to the current round of qps is less than 80% (preset value), let the CPU load corresponding to the current round of qps be current_cpu_workload, then the predicted CPU load corresponding to high_qps is min(task_conclusion_cpu_threshold,(1+current_cpu_workload) / 2); then based on the CPU load data and qps data after sorting the test results set, fit the correlation between CPU load and qps, determine the qps corresponding to the above filtered CPU load, and use it as the third qps for upward iteration. For example, using the existing CPU load data and QPS data in perf_data_list, the relationship between QPS and CPU can be fitted (linear fitting or binomial fitting can be used), and the high_qps corresponding to min(task_conclusion_cpu_threshold,(1+current_cpu_workload) / 2) can be obtained. If high_qps is less than current_qps (meaning that the fitting is inaccurate), then high_qps = current_qps + 4*qps_step.

[0127] In the embodiments of this specification, the above-mentioned sorting of the test result set data according to QPS and predicting the fourth QPS for the next iteration includes:

[0128] If the current timeout rate is greater than or equal to the preset timeout rate threshold, half of the current round's QPS will be determined as the fourth QPS for the next iteration.

[0129] If the current timeout rate is greater than or equal to the task stop timeout rate threshold and less than the preset timeout rate threshold, the fourth qps for the next iteration is obtained based on the difference between the current round qps and n times the unit iteration qps; n is greater than or equal to 2.

[0130] If the current timeout rate is less than the task stop timeout rate threshold, calculate the difference between the current round QPS and the unit iteration QPS to obtain the fourth QPS for the next iteration.

[0131] In the embodiments of this specification, if the QPS is iterated downwards, the following cases apply:

[0132] a) If the timeout rate corresponding to the current round's qps is greater than or equal to lower_bound_binary_search_error_rate, then lower_qps = current_qps / 2;

[0133] b) If the timeout rate corresponding to the current round QPS is greater than or equal to task_end_error_rate and less than lower_bound_binary_search_error_rate, then the fourth QPS for the next iteration is obtained based on the difference between the current round QPS and n times the above unit iteration QPS.

[0134] c) If the timeout rate corresponding to the current round's qps is less than task_end_error_rate, then lower_qps = current_qps - qps_step;

[0135] In the embodiments of this specification, the above-mentioned method of obtaining the fourth qps for downward iteration based on the difference between the current round qps and n times the unit iteration qps includes:

[0136] Calculate the difference between the current round's QPS and n times the QPS of the previous unit iteration to obtain the current QPS difference;

[0137] If the current QPS difference is greater than zero, the current QPS difference will be determined as the fourth QPS for the next iteration.

[0138] If the current QPS difference is less than zero, calculate the difference between the current round QPS and the unit iteration QPS.

[0139] If the difference between the current round's QPS and the unit iteration's QPS is greater than zero, then the difference is determined as the fourth QPS for the next iteration.

[0140] In the embodiments described in this specification, if the above difference is less than or equal to zero, the process returns directly and does not continue iterating. If the timeout rate corresponding to the current round's QPS is greater than or equal to `task_end_error_rate` and less than `lower_bound_binary_search_error_rate`, the difference between the current round's QPS and n times the above unit iteration QPS can be calculated to obtain the current QPS difference, i.e., `lower_qps = current_qps - 4 * qps_step`. If `current_qps - 4 * qps_step` is less than 0, then `lower_qps = current_qps - qps_step`. If `current_qps - qps_step <= 0`, the process returns directly.

[0141] In the embodiments described in this specification, such as Figure 7 As shown, the above method also includes:

[0142] S701: Get the current test result set for the current round;

[0143] S703: Based on the current test result set above, determine the first current timeout rate corresponding to the current maximum QPS and the second current timeout rate corresponding to the current minimum QPS;

[0144] S705: If the first current timeout rate is less than or equal to the task stop timeout rate threshold, iterate upwards based on the third qps mentioned above.

[0145] S707: If the second current timeout rate is less than or equal to the timeout rate threshold of the task conclusion, iterate downwards based on the fourth qps.

[0146] In the embodiments of this specification, it can be determined whether to iterate upwards or downwards in QPS based on the perf_data_list data that has been tested so far. If the timeout rate corresponding to the current maximum QPS is less than or equal to task_end_error_rate, then iterate upwards, and the QPS of the next round is the high_qps predicted in step 2). If the timeout rate corresponding to the current minimum QPS is greater than or equal to task_conclusion_error_rate, then iterate downwards, and the QPS of the next round is the lower_qps predicted above.

[0147] In the embodiments described in this specification, such as Figure 8 As shown, the above method also includes:

[0148] S801: Obtain the minimum QPS in the current test result set that is greater than or equal to the timeout rate threshold of the above task conclusion, and use it as the third current timeout rate;

[0149] S803: Obtain the maximum QPS in the current test result set that is less than the timeout rate threshold of the above task conclusion, and use it as the fourth current timeout rate;

[0150] S805: Calculate the difference between the above-mentioned third current timeout rate and the above-mentioned third current timeout rate to obtain the current timeout rate difference;

[0151] S807: If the current timeout rate difference is greater than the unit iteration qps, continue iterative testing using the above binary search method;

[0152] S809: If the current timeout rate difference is less than or equal to the unit iteration qps, the iteration is completed, and the qps at the time of iteration completion is determined as the target upper limit qps of the device.

[0153] In the embodiments of this specification, the target upper limit QPS of the device is the maximum QPS supported by a single device. If the current timeout rate difference is greater than the unit iteration QPS, the average value of the current timeout rate difference is calculated using the binary search method as the QPS for the next round, and iterative testing is performed. If it is in the middle of the iteration, the minimum QPS with a timeout rate greater than or equal to task_conclusion_error_rate is obtained from the perf_data_list data and set as qps2 (i.e., the third current timeout rate), and the maximum QPS with a timeout rate less than task_conclusion_error_rate is set as qps1 (the fourth current timeout rate). If qps2-qps1>qps_step, the QPS needs to be iterated using the binary search method, and the QPS for the next round is (qps2-qps1) / 2. If qps2-qps1<=qps_step, the QPS iteration is completed, and the task is exited.

[0154] In one exemplary embodiment, such as Figure 9-11 As shown, the method of this embodiment can be applied to a server performance testing platform; as Figure 9 As shown, Figure 9 This is a display interface for a server performance testing platform, which includes a task creation control: "+Create Task". Triggering this control will redirect to... Figure 10 The task creation page shown allows you to trigger the performance testing method of this embodiment by selecting "Yes" for the filter control corresponding to "smart qps". The selection of the "smart qps" parameter indicates that the automated performance testing method of this invention is applied to iterate the qps, quickly finding the server's maximum qps. Figure 11 The performance test results shown can be used for subsequent resource quantity assessments and other purposes.

[0155] In the embodiments described in this specification, the method of this embodiment achieves a test success rate of over 99% for the device, which is significantly higher than that of existing testing solutions, and the number of test iterations is reduced by 30%.

[0156] Specifically, in the embodiments of this specification, such as Figure 12 As shown, the performance testing method for the device in this embodiment includes:

[0157] 1. Obtain the qps of the 5 most recent performance data within the historical time period, and get the maximum qps as max_qps and the minimum qps as min_qps;

[0158] 2. Use min_qps for the first round of load testing to obtain the corresponding CPU usage and first-round timeout rate (error_rate);

[0159] 3. Determine if the error rate corresponding to min_qps is greater than or equal to 10%;

[0160] 4. If yes, perform load testing using min_qps / 2; proceed to step 6; if no, determine whether the error_rate corresponding to min_qps is greater than or equal to end_error_rate;

[0161] 5. If not, use max_qps for load testing; if yes, use min_qps-4*qps_step for load testing.

[0162] 6. Collect the (qps, error_rate, cpu) 3-tuples from previous rounds and sort them by qps;

[0163] 7. Determine if certain rounds of QPS need to be rerun; if so, further determine if the number of reruns exceeds the configured value (default is 1); if not, rerun the previous two QPS rounds.

[0164] 8. If there are no reruns, determine whether the iteration QPS is in the middle;

[0165] 9. If there is an intermediate iteration QPS, calculate the upper and lower bounds of the intermediate iteration QPS; determine whether the difference between the upper and lower bound QPS is greater than qps_step; if not, exit normally; if yes, calculate half of the sum of the upper and lower bound QPS and use that QPS for load testing.

[0166] 10. If the QPS is not iterated in the middle, determine whether to iterate upwards in QPS;

[0167] 11. If iterating upwards in QPS, determine the CPU load corresponding to the most recent QPS.

[0168] Is it greater than or equal to 80%?

[0169] 12. If the CPU load corresponding to the most recent round of QPS is greater than or equal to 80%, determine whether the system failure rate corresponding to the most recent round of QPS is greater than or equal to conclusion_error_rate; if yes, use the most recent round of QPS + qps_step for load testing; if no, use the most recent round of QPS + 4 * qps_step for load testing.

[0170] 13. If the CPU load corresponding to the most recent round of QPS is less than 80%, find half of the sum of the current round of CPU load and 100%, take the smaller value between the current round of CPU load and 80% of CPU load, and set it as predict_cpu. Use the performance data of the previous rounds to fit the QPS corresponding to predict_cpu, set it as predict_qps, and use predict_qps for stress testing.

[0171] 14. If not iterating upwards in QPS, determine whether the system failure rate corresponding to the most recent QPS is greater than 10%;

[0172] 15. If yes, use half of the most recent QPS for load testing; if no, determine whether the system failure rate corresponding to the most recent QPS is less than end_error_rate.

[0173] 16. If yes, use the most recent QPS minus qps_step for load testing; if no, use the most recent QPS minus 4*qps_step for load testing.

[0174] As can be seen from the technical solutions provided in the embodiments of this specification above, the embodiments of this specification obtain a preset upper limit threshold and a preset lower limit threshold for the server; test the server based on the preset lower limit threshold to obtain initial test results; the initial test results include an initial timeout rate; if the initial timeout rate is greater than or equal to the preset timeout rate threshold, the lower limit threshold is iterated using a binary search method to obtain the next round of QPS threshold; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than a first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test; if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold, the next round of QPS threshold is determined based on the lower limit threshold and the unit iteration QPS; if the initial timeout rate is less than the task conclusion timeout rate threshold, the next round of QPS threshold is determined based on the preset upper limit threshold; thus, the next round of QPS threshold can be quickly and accurately determined under different conditions, and the server can be tested based on the next round of QPS threshold to obtain performance test results. This application enables the setting of different QPS thresholds for the next round based on different situations, reducing the number of iterations in the performance testing process and enabling the rapid determination of device performance test results, thereby improving testing efficiency.

[0175] This specification also provides an embodiment of a device performance testing apparatus, such as... Figure 13 As shown, the device includes:

[0176] The limit threshold acquisition module 1310 is used to acquire the preset upper limit threshold and the preset lower limit threshold of the device;

[0177] The initial result determination module 1320 is used to test the device based on the preset qps lower limit threshold and obtain initial test results; the initial test results include the initial timeout rate;

[0178] The first threshold determination module 1330 is used to obtain the next round of QPS threshold by iterating the lower limit threshold of QPS using a binary search method if the initial timeout rate is greater than or equal to a preset timeout rate threshold; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than the first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test process;

[0179] The second threshold determination module 1340 is used to determine the next round qps threshold based on the qps lower limit threshold and the unit iteration qps if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold.

[0180] The third threshold determination module 1350 is used to determine the next round qps threshold based on the preset qps upper limit threshold if the initial timeout rate is less than the task conclusion timeout rate threshold.

[0181] The test result determination module 1360 is used to test the device based on the next round of qps threshold and obtain performance test results.

[0182] In some embodiments, the second threshold determining module includes:

[0183] The upward iteration threshold determination unit is used to determine the next round of qps threshold by iterating upward based on the lower limit threshold of qps and the unit iteration qps if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the task stop timeout rate threshold; the difference between the task stop timeout rate threshold and the task conclusion timeout rate threshold is less than a second threshold, and the task stop timeout rate threshold is less than the preset timeout rate threshold.

[0184] The downward iteration threshold determination unit is used to determine the next round of qps threshold by iterating downward based on the lower limit threshold of qps and the unit iteration qps if the initial timeout rate is greater than or equal to the task stop timeout rate threshold and less than the preset timeout rate threshold.

[0185] In some embodiments, the test result determination module includes:

[0186] The timeout rate determination unit is used to test the device based on the next round QPS threshold to obtain a first timeout rate and a second timeout rate, wherein the first timeout rate is greater than the task conclusion timeout rate threshold; and the second timeout rate is less than or equal to the task conclusion timeout rate threshold.

[0187] The first QPS acquisition unit is used to acquire the minimum QPS of the first timeout rate during the test process, and obtain the first QPS;

[0188] The second qps acquisition unit is used to acquire the maximum qps of the second timeout rate during the test process, and obtain the second qps.

[0189] The target difference determination unit is used to calculate the difference between the first qps and the second qps to obtain the target difference;

[0190] A stopping unit is used to trigger a test stop if the target difference is less than or equal to the unit iteration qps.

[0191] The result determination unit is used to continue the next round of testing if the target difference is greater than the unit iteration qps, and obtain the performance test result.

[0192] In some embodiments, the apparatus further includes:

[0193] The judgment module is used to determine whether there are any rounds that meet the rerun mechanism when the number of test rounds exceeds a preset threshold.

[0194] The rerun module is used to determine the target QPS corresponding to the rerun mechanism if there are rounds that satisfy the rerun mechanism, and to perform a rerun test based on the target QPS.

[0195] The prediction module is used to obtain the test result set of each round of testing of the device if there is no round that satisfies the rerun mechanism, sort the data in the test result set according to QPS, and predict the third QPS for upward iteration and the fourth QPS for downward iteration.

[0196] In some embodiments, the determining module includes:

[0197] The third timeout rate determination unit is used to test the device based on the third QPS to obtain the third timeout rate;

[0198] The fourth timeout rate determination unit is used to test the device based on the fourth qps to obtain the fourth timeout rate;

[0199] The first determining unit is used to determine that there is a round that satisfies the rerun mechanism if the third timeout rate is less than the task conclusion timeout rate threshold and the fourth timeout rate is greater than the task conclusion timeout rate threshold.

[0200] The second determining unit is used to determine that there are no rounds that satisfy the rerun mechanism if the third timeout rate is greater than or equal to the task conclusion timeout rate threshold, and / or the fourth timeout rate is less than or equal to the task conclusion timeout rate threshold.

[0201] In some embodiments, the rerun module includes:

[0202] The absolute difference calculation unit is used to calculate the absolute difference between the third timeout rate and the task conclusion timeout rate threshold, and to calculate the absolute difference between the fourth timeout rate and the task conclusion timeout rate threshold.

[0203] The first target QPS determination unit is used to determine the absolute value of the first difference as the target QPS if the absolute value of the first difference is less than or equal to the absolute value of the second difference.

[0204] The second target QPS determination unit is used to determine the second absolute value of the difference as the target QPS if the absolute value of the first difference is greater than the absolute value of the second difference.

[0205] In some embodiments, the apparatus further includes:

[0206] The QPS re-determination module is used to re-determine the highest QPS in the rerun test process as the third QPS and the lowest QPS in the rerun test process as the fourth QPS when the number of reruns is less than or equal to a preset threshold number of reruns.

[0207] The jump module is used to jump to the step of testing the device based on the third QPS to obtain the third timeout rate;

[0208] The test exit module is used to exit the rerun test when the number of reruns exceeds a preset threshold.

[0209] In some embodiments, the performance test results include CPU load and timeout rate, and the prediction module includes:

[0210] The current timeout rate acquisition unit is used to obtain the current CPU load and current timeout rate corresponding to the QPS in the current round;

[0211] The third QPS prediction unit is used to calculate the sum of the current round QPS and the unit iteration QPS if the current CPU load is greater than or equal to a preset value and the current timeout rate is greater than or equal to the task conclusion timeout rate threshold, so as to obtain the third QPS for upward iteration.

[0212] The fourth QPS prediction unit is used to sort the data in the test result set according to QPS and predict the fourth QPS for the next iteration.

[0213] In some embodiments, the apparatus further includes:

[0214] The third QPS calculation module is used to calculate the sum of the current round QPS and n times the unit iteration QPS if the current CPU load is greater than or equal to the preset value and the current timeout rate is less than the task conclusion timeout rate threshold, to obtain the third QPS for upward iteration; n is greater than or equal to 2.

[0215] The target CPU load determination module is used to predict the target CPU load based on the current CPU load and a preset CPU load threshold if the current CPU load is less than the preset value.

[0216] The third QPS determination module is used to fit the correlation between CPU load and QPS based on the CPU load data and QPS data sorted in the test result set, and to determine the third QPS for upward iteration based on the correlation and the target CPU load.

[0217] In some embodiments, the third QPS determination module includes:

[0218] The target load threshold determination unit is used to calculate the average value of the target CPU load and the target value to obtain the target load threshold;

[0219] A threshold comparison unit is used to compare the target load threshold with a preset CPU load threshold, and to determine the smaller value between the target load threshold and the preset CPU load threshold as the filtered CPU load.

[0220] The third QPS determination unit is used to determine the QPS corresponding to the filtered CPU load based on the correlation relationship, and use it as the third QPS for upward iteration.

[0221] In some embodiments, the fourth qps prediction unit includes:

[0222] The first prediction subunit is used to determine half of the current round's qps as the fourth qps for the next iteration if the current timeout rate is greater than or equal to the preset timeout rate threshold.

[0223] The second prediction subunit is used to obtain the fourth qps for the downward iteration based on the difference between the current round qps and n times the unit iteration qps if the current timeout rate is greater than or equal to the task stop timeout rate threshold and less than the preset timeout rate threshold; n is greater than or equal to 2.

[0224] The third prediction subunit is used to calculate the difference between the current round QPS and the unit iteration QPS if the current timeout rate is less than the task stop timeout rate threshold, and obtain the fourth QPS for the next iteration.

[0225] In some embodiments, the second prediction subunit includes:

[0226] The current qps difference determination subunit is used to calculate the difference between the current round qps and n times the unit iteration qps to obtain the current qps difference;

[0227] The fourth qps determination subunit is used to determine the current qps difference as the fourth qps for the next iteration if the current qps difference is greater than zero.

[0228] The iterative difference calculation subunit is used to calculate the difference between the current round qps and the unit iteration qps if the current qps difference is less than zero.

[0229] The fourth QPS prediction subunit is used to determine the difference as the fourth QPS for the next iteration if the difference between the current round QPS and the unit iteration QPS is greater than zero.

[0230] In some embodiments, the apparatus further includes:

[0231] The current result set determination module is used to obtain the current test result set for the current round;

[0232] The current timeout rate determination module is used to determine the first current timeout rate corresponding to the current maximum qps and the second current timeout rate corresponding to the current minimum qps based on the current test result set.

[0233] The first test module is used to iterate upwards based on the third qps if the first current timeout rate is less than or equal to the task stop timeout rate threshold.

[0234] The second testing module is used to iterate downwards based on the fourth QPS if the second current timeout rate is less than or equal to the task conclusion timeout rate threshold.

[0235] In some embodiments, the apparatus further includes:

[0236] The third current timeout rate acquisition module is used to obtain the minimum QPS in the current test result set that is greater than or equal to the task conclusion timeout rate threshold, as the third current timeout rate;

[0237] The fourth current timeout rate acquisition module is used to acquire the maximum QPS where the timeout rate in the current test result set is less than the task conclusion timeout rate threshold, and use it as the fourth current timeout rate.

[0238] The current timeout rate difference calculation module is used to calculate the difference between the third current timeout rate and the third current timeout rate to obtain the current timeout rate difference.

[0239] The iterative testing module is used to continue iterative testing using the binary search method if the current timeout rate difference is greater than the unit iteration qps.

[0240] The target upper limit qps determination module is used to determine the target upper limit qps of the device if the current timeout rate difference is less than or equal to the unit iteration qps, the iteration is completed, and the qps at the time of iteration completion is determined as the target upper limit qps of the device.

[0241] The apparatus and method embodiments described herein are based on the same inventive concept.

[0242] This specification provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the performance testing method of the device as provided in the above method embodiments.

[0243] Embodiments of this application also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to implementing a performance testing method of a device in the method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the performance testing method of the device provided in the above method embodiment.

[0244] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the performance testing method for the device provided in the above-described method embodiments.

[0245] Optionally, in the embodiments of this specification, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0246] The memory described in the embodiments of this specification can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for the functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0247] The device performance testing methods provided in the embodiments of this specification can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Taking running on a server as an example, Figure 14 This is a hardware structure block diagram of a server for a device performance testing method provided in an embodiment of this specification. (Example:) Figure 14As shown, the server 1400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1410 (CPUs 1410 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 1430 for storing data, and one or more storage media 1420 (e.g., one or more mass storage devices) for storing application programs 1423 or data 1422. The memory 1430 and storage media 1420 may be temporary or persistent storage. The program stored in the storage media 1420 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 1410 may be configured to communicate with the storage media 1420 and execute the series of instruction operations stored in the storage media 1420 on the server 1400. Server 1400 may also include one or more power supplies 1460, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1440, and / or one or more operating systems 1421, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0248] The input / output interface 1440 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 1400. In one example, the input / output interface 1440 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In one example, the input / output interface 1440 can be a radio frequency (RF) module for wireless communication with the Internet.

[0249] Those skilled in the art will understand that Figure 14 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 1400 may also include... Figure 14 The more or fewer components shown, or having the same Figure 14 The different configurations shown.

[0250] As can be seen from the embodiments of the performance testing method, apparatus, device, or storage medium provided in this application, this application obtains a preset upper limit threshold and a preset lower limit threshold for the server; tests the server based on the preset lower limit threshold to obtain initial test results; the initial test results include an initial timeout rate; if the initial timeout rate is greater than or equal to a preset timeout rate threshold, iterates the lower limit threshold using a binary search method to obtain the next round of QPS threshold; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than the threshold value of the first round. A threshold is set; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test process; if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold, and less than the preset timeout rate threshold, the next round QPS threshold is determined based on the lower limit threshold of QPS and the QPS per unit iteration; if the initial timeout rate is less than the task conclusion timeout rate threshold, the next round QPS threshold is determined based on the preset upper limit threshold of QPS; thus, the next round QPS threshold under different conditions can be quickly and accurately determined, and the server can be tested based on the next round QPS threshold to obtain performance test results. This application realizes the setting of different next round QPS thresholds according to different situations, reduces the number of iterations in the performance test process, and can quickly determine the performance test results of the device, thus improving test efficiency.

[0251] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0252] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0253] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0254] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for testing the performance of a device, characterized in that, The method includes: Obtain the device's preset upper limit threshold and preset lower limit threshold for QPS; The device is tested based on the preset lower limit threshold qps to obtain initial test results; the initial test results include the initial timeout rate. If the initial timeout rate is greater than or equal to the preset timeout rate threshold, the QPS lower limit threshold is iterated using a binary search method to obtain the QPS threshold for the next round; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than a first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test process; If the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold, the next round of QPS threshold is determined based on the QPS lower limit threshold and the unit iteration QPS. If the initial timeout rate is less than the task conclusion timeout rate threshold, the next round of qps threshold is determined based on the preset qps upper limit threshold. The device is tested based on the next round of QPS threshold to obtain performance test results.

2. The method according to claim 1, characterized in that, If the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold, and less than the preset timeout rate threshold, the next round QPS threshold is determined based on the QPS lower limit threshold and the unit iteration QPS, including: If the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the task stop timeout rate threshold, the next round of QPS threshold is determined by iterating upward based on the lower limit threshold of QPS and the unit iteration QPS; the difference between the task stop timeout rate threshold and the task conclusion timeout rate threshold is less than the second threshold, and the task stop timeout rate threshold is less than the preset timeout rate threshold. If the initial timeout rate is greater than or equal to the task stop timeout rate threshold, but less than the preset timeout rate threshold, the next round of QPS threshold is determined by iterating downwards based on the lower limit threshold of QPS and the unit iteration QPS.

3. The method according to claim 1, characterized in that, The process of testing the device based on the next round of QPS threshold to obtain performance test results includes: The device is tested based on the next round QPS threshold to obtain a first timeout rate and a second timeout rate. The first timeout rate is greater than the task conclusion timeout rate threshold; the second timeout rate is less than or equal to the task conclusion timeout rate threshold. Obtain the minimum QPS of the first timeout rate during the test process to get the first QPS; Obtain the maximum QPS of the second timeout rate during the test process to get the second QPS; Calculate the difference between the first QPS and the second QPS to obtain the target difference; If the target difference is less than or equal to the unit iteration qps, then the test is stopped. If the target difference is greater than the unit iteration qps, continue to execute the next round of testing to obtain the performance test results.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: If the number of test rounds exceeds a preset threshold, determine whether there are any rounds that meet the rerun mechanism; If there are rounds that satisfy the rerun mechanism, determine the target QPS corresponding to the rerun mechanism, and perform the rerun test based on the target QPS; If there is no round that satisfies the rerun mechanism, obtain the test result set of each round of testing for the device, sort the data in the test result set according to QPS, and predict the third QPS for upward iteration and the fourth QPS for downward iteration.

5. The method according to claim 4, characterized in that, The determination of whether there is a round that satisfies the rerun mechanism includes: The device is tested based on the third QPS to obtain the third timeout rate; The device is tested based on the fourth QPS to obtain the fourth timeout rate; If the third timeout rate is less than the task conclusion timeout rate threshold, and the fourth timeout rate is greater than the task conclusion timeout rate threshold, it is determined that there is a round that satisfies the rerun mechanism. If the third timeout rate is greater than or equal to the task conclusion timeout rate threshold, and / or the fourth timeout rate is less than or equal to the task conclusion timeout rate threshold, it is determined that there is no round that satisfies the rerun mechanism.

6. The method according to claim 5, characterized in that, The target QPS corresponding to the determination of the rerun mechanism includes: Calculate the absolute value of the first difference between the third timeout rate and the task conclusion timeout rate threshold, and calculate the absolute value of the second difference between the fourth timeout rate and the task conclusion timeout rate threshold; If the absolute value of the first difference is less than or equal to the absolute value of the second difference, the absolute value of the first difference is determined as the target QPS; If the absolute value of the first difference is greater than the absolute value of the second difference, the absolute value of the second difference is determined as the target QPS.

7. The method according to claim 5, characterized in that, The method further includes: When the number of retries is less than or equal to a preset threshold, the highest QPS during the retries test is redefined as the third QPS, and the lowest QPS during the retries test is redefined as the fourth QPS. Jump to the step of testing the device based on the third QPS to obtain the third timeout rate; When the number of retries exceeds a preset threshold, the retries test is terminated.

8. The method according to claim 4, characterized in that, The performance test results include CPU load and timeout rate. The step of sorting the test results by QPS and predicting the third QPS for upward iteration and the fourth QPS for downward iteration includes: Get the current CPU load and current timeout rate corresponding to the QPS of the current round; If the current CPU load is greater than or equal to a preset value, and the current timeout rate is greater than or equal to the task conclusion timeout rate threshold, calculate the sum of the current round QPS and the unit iteration QPS to obtain the third QPS for upward iteration; Sort the test result set data according to QPS and predict the fourth QPS for the next iteration.

9. The method according to claim 8, characterized in that, The method further includes: If the current CPU load is greater than or equal to the preset value, and the current timeout rate is less than the task conclusion timeout rate threshold, calculate the sum of the current round QPS and n times the unit iteration QPS to obtain the third QPS for upward iteration; n is greater than or equal to 2; If the current CPU load is less than the preset value, predict the target CPU load based on the current CPU load and the preset CPU load threshold; Based on the CPU load data and QPS data after sorting the test results, the correlation between CPU load and QPS is obtained by fitting, and the third QPS for upward iteration is determined based on the correlation and the target CPU load.

10. The method according to claim 9, characterized in that, The process of determining the third QPS for upward iteration based on the correlation and the target CPU load includes: Calculate the average of the target CPU load and the target value to obtain the target load threshold; Compare the target load threshold with the preset CPU load threshold, and determine the smaller value between the target load threshold and the preset CPU load threshold as the filtered CPU load. Based on the aforementioned correlation, the QPS corresponding to the selected CPU load is determined and used as the third QPS for upward iteration.

11. The method according to claim 8, characterized in that, The step of sorting the test result set data according to QPS and predicting the fourth QPS for the next iteration includes: If the current timeout rate is greater than or equal to the preset timeout rate threshold, half of the current round's QPS is determined as the fourth QPS for the next iteration; If the current timeout rate is greater than or equal to the task stop timeout rate threshold and less than the preset timeout rate threshold, the fourth qps for the next iteration is obtained based on the difference between the current round qps and n times the unit iteration qps; n is greater than or equal to 2. If the current timeout rate is less than the task stop timeout rate threshold, calculate the difference between the current round QPS and the unit iteration QPS to obtain the fourth QPS for the next iteration.

12. The method according to claim 11, characterized in that, The step of obtaining the fourth QPS for the downward iteration based on the difference between the current round's QPS and n times the unit iteration's QPS includes: Calculate the difference between the current round's QPS and n times the unit iteration QPS to obtain the current QPS difference; If the current QPS difference is greater than zero, the current QPS difference is determined as the fourth QPS for the next iteration; If the current QPS difference is less than zero, calculate the difference between the current round QPS and the unit iteration QPS; If the difference between the current round's QPS and the unit iteration's QPS is greater than zero, the difference is determined as the fourth QPS for the next iteration.

13. The method according to claim 8, characterized in that, The method further includes: Get the current test result set for the current round; Based on the current test result set, determine the first current timeout rate corresponding to the current maximum QPS and the second current timeout rate corresponding to the current minimum QPS; If the first current timeout rate is less than or equal to the task stop timeout rate threshold, a test is performed based on the third QPS of the upward iteration; If the second current timeout rate is less than or equal to the task conclusion timeout rate threshold, a test is performed based on the fourth QPS of the downward iteration.

14. The method according to claim 13, characterized in that, The method further includes: The minimum QPS in the current test result set that is greater than or equal to the timeout rate threshold of the task conclusion is obtained as the third current timeout rate; The maximum QPS where the timeout rate in the current test result set is less than the timeout rate threshold of the task conclusion is obtained as the fourth current timeout rate. Calculate the difference between the third current timeout rate and the third current timeout rate to obtain the current timeout rate difference; If the current timeout rate difference is greater than the unit iteration qps, continue iterative testing using the binary search method; If the current timeout rate difference is less than or equal to the unit iteration qps, the iteration is completed, and the qps at the time of iteration completion is determined as the target upper limit qps of the device.

15. A performance testing device for equipment, characterized in that, The device includes: The limit threshold acquisition module is used to acquire the preset upper limit threshold and the preset lower limit threshold of the device's QPS. The initial result determination module is used to test the device based on the preset lower limit threshold of qps and obtain initial test results; the initial test results include the initial timeout rate; The first threshold determination module is used to obtain the next round of QPS threshold by iterating the lower limit threshold of QPS using a binary search method if the initial timeout rate is greater than or equal to a preset timeout rate threshold; the preset timeout rate threshold is determined based on the task conclusion timeout rate threshold, and the difference between the preset timeout rate threshold and the task conclusion timeout rate threshold is greater than the first threshold; the test QPS corresponding to the task conclusion timeout rate threshold is the maximum QPS during the test process; The second threshold determination module is used to determine the next round qps threshold based on the qps lower limit threshold and the unit iteration qps if the initial timeout rate is greater than or equal to the task conclusion timeout rate threshold and less than the preset timeout rate threshold. The third threshold determination module is used to determine the next round qps threshold based on the preset qps upper limit threshold if the initial timeout rate is less than the task conclusion timeout rate threshold. The test result determination module is used to test the device based on the next round of qps threshold and obtain performance test results.

Images