Performance test method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of computers, and provides a performance test method and device, equipment, a storage medium and a program product. The method comprises the following steps: publishing a to-be-tested version code of a business system to a test environment; obtaining interface service information, and generating a basic task based on the interface service information; based on a multi-level index judgment rule, modeling processing is performed on the basic task, and a performance pressure measurement task is generated; and based on the performance pressure test task, performing performance test on the to-be-tested version code to obtain performance test result data. By means of the mode, performance testing does not need to be carried out in a production environment, resources identical to the production environment do not need to be configured for a testing environment, the generated performance pressure testing task only needs to be executed in the fixed testing environment, the performance testing result data of the to-be-tested version code can be obtained, dependence of performance testing on tested environment configuration is shielded, and the testing efficiency of the to-be-tested version code is improved. Therefore, resource waste is avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a performance testing method, device, equipment, storage medium and program product. Background Art

[0002] As agile development models prevail, the iteration speed of business system code versions has accelerated, so regular performance testing is needed to ensure that the code performance of each version can meet the expected standards to meet the needs of rapid launch and continuous optimization of business systems.

[0003] Performance testing can be used to predict the performance status of the business system in advance, discover potential performance problems in a timely manner, and avoid business system failures after they are opened to users.

[0004] There are two existing performance testing methods: one is to perform performance testing on the new version of the code directly in the production environment, but this method is likely to pollute the real business data of the business system; the other is to configure the same resources as the production environment in the test environment, and then perform performance testing on the new version of the code in the test environment, but when the resource configuration of the production environment is high, this method is likely to cause resource waste.

[0005] Therefore, existing performance testing methods have high requirements on environmental configuration and easily lead to waste of resources. Summary of the invention

[0006] The embodiments of the present application provide a performance testing method, apparatus, device, storage medium and program product to solve the technical problem that the existing performance testing method has high requirements on environmental configuration and easily leads to waste of resources.

[0007] In a first aspect, an embodiment of the present application provides a performance testing method, including: publishing a version code to be tested of a business system to a test environment; obtaining interface service information, and generating basic tasks based on the interface service information; modeling the basic tasks based on multi-level indicator judgment rules to generate performance stress testing tasks; based on the performance stress testing tasks, performing performance testing on the version code to be tested to obtain performance testing result data.

[0008] In one embodiment, based on the multi-level indicator judgment rule, the basic task is modeled and processed to generate a performance stress test task, including: executing the basic task to obtain the basic task execution result; the basic task execution result includes the first-level indicator, the second-level indicator and the third-level indicator, the first-level indicator includes the abnormality rate of each interface in the business system, the second-level indicator includes the overall resource utilization rate of the business system, and the third-level indicator includes the TPS value and the first time consumption in each thread group test result; judging whether the abnormality rate of each interface meets the first target requirement; if the abnormality rate of each interface meets the first target requirement, judging whether the overall resource utilization rate meets the second target requirement; if the overall resource utilization rate meets the second target requirement, If the rate does not meet the second target requirement, the basic task is adjusted to obtain the adjusted basic task, and the step of returning to execute the basic task and obtaining the execution result of the basic task is performed until the overall resource utilization rate meets the second target requirement; it is determined whether the TPS value and the first time consumption in the test result of each thread group meet the third target requirement; if the TPS value and the first time consumption in the test result of each thread group do not meet the third target requirement, the basic task is adjusted to obtain the adjusted basic task, and the step of returning to execute the basic task and obtaining the execution result of the basic task is performed until the TPS value and the first time consumption in the test result of each thread group meet the third target requirement, and a performance stress testing task is generated.

[0009] In one embodiment, the performance test result data includes system-level indicator data and business-level indicator data; based on the performance stress testing task, a performance test is performed on the version code to be tested, and after obtaining the performance test result data, it also includes: obtaining historical performance test result data of historical version codes; the historical performance test result data includes historical system-level indicator data and historical business-level indicator data; based on the business-level indicator data and the historical business-level indicator data, generating and displaying business-level indicator analysis results of the version code to be tested; based on the system-level indicator data and the historical system-level indicator data, generating and displaying system-level indicator analysis results of the version code to be tested; based on the business-level indicator analysis results and the system-level indicator analysis results, generating and displaying a performance trend curve.

[0010] In one embodiment, the business-level indicator data includes performance parameters of each interface in the business system, the TPS value of the interface service, the throughput, the average response time, the first response time, the second response time, the third response time, the maximum response time, the exception rate and the number of requests, the first response time is the response time when processing 90% of the requests, the second response time is the response time when processing 95% of the requests, and the third response time is the response time when processing 99% of the requests. The business-level indicator analysis results include execution comparison results and exception comparison results; based on the business-level indicator data and the historical business-level indicator data, the business-level indicator analysis results of the version code to be tested are generated and displayed, including: comparing and analyzing each performance parameter with the performance baseline, generating and displaying the execution comparison results; calculating the exception data based on the business-level indicator data and the historical business-level indicator data, and generating and displaying the exception comparison results.

[0011] In one embodiment, the system-level indicator data includes CPU consumption data, memory consumption data, middleware consumption data and resource utilization rate; based on the system-level indicator data and historical system-level indicator data, the system-level indicator analysis results of the version code to be tested are generated and displayed, including: comparing and analyzing the system-level indicator data and the historical system-level indicator data to determine the resource consumption difference between the version code to be tested and the historical version code; based on the resource consumption difference, the system-level indicator analysis results of the version code to be tested are generated and displayed.

[0012] In one embodiment, interface service information is obtained, and basic tasks are generated based on the interface service information, including: obtaining interface service information through an interface platform and a log platform; calculating an initial TPS value, an initial timer setting number, and an initial concurrent number setting value based on the interface service information; and generating basic tasks based on the initial TPS value, the initial timer setting number, and the initial concurrent number setting value.

[0013] In the second aspect, an embodiment of the present application provides a performance testing device, including: a publishing module, used to publish the tested version code of the business system to the test environment; a basic task generation module, used to obtain interface service information and generate basic tasks based on the interface service information; a stress testing task generation module, used to model the basic tasks based on multi-level indicator judgment rules and generate performance stress testing tasks; a testing module, used to perform performance testing on the tested version code based on the performance stress testing tasks and obtain performance test result data.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned performance testing methods when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned performance testing methods.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any of the above-mentioned performance testing methods when executed by a processor.

[0017] The performance testing method, apparatus, equipment, storage medium and program product provided in the embodiments of the present application publish the tested version code of the business system to the test environment; obtain interface service information, and generate basic tasks based on the interface service information; model the basic tasks based on multi-level indicator judgment rules to generate performance stress testing tasks; based on the performance stress testing tasks, the tested version code is performance tested to obtain performance test result data. Through the above method, there is no need to perform performance testing in the production environment, nor is there any need to configure the test environment with the same resources as the production environment. It is only necessary to execute the generated performance stress testing tasks in a fixed test environment to obtain the performance test result data of the tested version code, shielding the performance test's dependence on the tested environment configuration, thereby avoiding resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the performance testing method provided in the embodiment of the present application.

[0020] Figure 2 This is the second flow chart of the performance testing method provided in the embodiment of the present application.

[0021] Figure 3 This is the third flow chart of the performance testing method provided in the embodiment of the present application.

[0022] Figure 4 It is a flowchart of the modeling processing provided in the embodiment of the present application.

[0023] Figure 5 It is a flowchart of the intelligent comparative analysis provided in the embodiment of the present application.

[0024] Figure 6 It is a schematic diagram of the structure of the performance testing device provided in the embodiment of the present application.

[0025] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] See also Figure 1 , Figure 1 It is one of the flow charts of the performance testing method provided in the embodiment of the present application. In the embodiment of the present application, the performance testing method is applied to the performance monitoring device, and the performance testing method includes steps S110 to S140, and each step is as follows: S110: Release the code of the business system to be tested to the test environment.

[0028] See also Figure 2 , Figure 2 This is the second flow chart of the performance testing method provided in the embodiment of the present application.

[0029] Specifically, Figure 2 As shown, the code in the production environment is the current version code of the business system, recorded as the current version code V1; when the code of the business system needs to be iteratively updated, the new version code V2 can be used as the version code to be tested.

[0030] It can be understood that in order to avoid contamination of real business data in the production environment, in this embodiment, it is necessary to release the tested version code of the business system to the test environment, and the performance monitoring device will perform performance testing on the tested version code in the test environment. If the tested version code can pass the performance test, and it can be determined based on the performance test result data that the tested version code meets actual production needs (that is, meets user needs), then the tested version code can be released to the production environment to achieve iterative updates of the business system.

[0031] S120: Acquire interface service information, and generate basic tasks based on the interface service information.

[0032] See also Figure 3 , Figure 3 This is the third flow chart of the performance testing method provided in the embodiment of the present application.

[0033] like Figure 3As shown, the performance monitoring device can obtain interface service information through the interface platform and the log platform, and automatically generate an interface performance test script, thereby eliminating the need for manual statistical calculations and the work of writing performance test scripts.

[0034] Specifically, the performance monitoring device can automatically obtain interface service information such as interface name, service path URL and other request header information, interface input parameter message, call volume, number of service node pods, etc. of all interfaces in the business system within a preset period through the interface platform, log platform and monitoring platform, and automatically calculate the initial TPS value, initial timer setting number and initial concurrency setting value based on the interface TPS (Transaction Per Second) value calculation formula.

[0035] Optionally, the calculation formula of the initial TPS value is as follows: Initial TPS value = total production calls / call statistics time / X; Where X represents the number of service node pods, and the call statistics time is in seconds.

[0036] Optionally, the initial concurrency setting value is obtained by rounding up the initial TPS value.

[0037] Furthermore, a basic task is generated based on the initial TPS value, the initial timer setting number and the initial concurrency setting value.

[0038] Preferably, the target requirements of the performance stress testing task are: the overall resource utilization cpu_limit of the business system needs to be between 20% and 80%, the first time consumption (that is, the time consumption for processing 90% of the tasks in each thread group test result) is less than 2000ms, and the abnormal rate is 0%.

[0039] It is understandable that the target requirements of the performance stress testing task can be modified by those skilled in the art according to actual testing requirements.

[0040] S130: Based on the multi-level indicator judgment rules, the basic tasks are modeled and a performance stress testing task is generated.

[0041] In the prior art, performance stress testing tasks are formulated by performance testers, and the final performance stress testing tasks are obtained through continuous adjustment through multiple executions. This approach will undoubtedly increase labor costs and time costs.

[0042] Based on this, in this embodiment, a modeling processing function is configured in the performance monitoring device, and performance stress testing tasks can be automatically generated through model processing, thereby solving the problem that performance stress testing tasks in the prior art require a lot of manpower.

[0043] Specifically, the model processing rules are multi-level indicator judgment rules. Based on the multi-level indicator judgment rules, the basic tasks can be modeled and processed, and performance stress testing tasks suitable for performance testing can be automatically generated, thereby reducing the manpower and time costs of executing strategies and executing stress testing.

[0044] It should be noted that, in this embodiment, the performance monitoring device can collect actual business data generated by the production environment through the interface platform, the log platform and the monitoring platform, and generate basic tasks based on the collected production environment data, and then model the basic tasks based on the multi-level indicator judgment rules to generate performance stress testing tasks. Under normal circumstances, the performance stress testing tasks do not change, that is, each version of the code uses the same performance stress testing tasks.

[0045] Preferably, in actual applications, data from the peak hours of user traffic in the production environment can be selected to generate basic tasks, so that tasks that can meet the performance requirements of the peak hours of user traffic will inevitably meet the performance requirements of the general hours. Performance stress testing tasks generated by modeling the basic tasks during the peak hours are more effective for performance testing in the test environment.

[0046] Preferably, the basic tasks and the performance stress testing tasks can be regenerated according to a preset period, and in the same period, only the same performance stress testing task needs to be generated and used.

[0047] Optionally, the preset period may be one month, one quarter, etc.

[0048] S140: Based on the performance stress testing task, a performance test is performed on the code of the tested version to obtain performance test result data.

[0049] The performance testing method provided in the embodiment of the present application publishes the tested version code of the business system to the test environment; obtains interface service information, and generates basic tasks based on the interface service information; models the basic tasks based on multi-level indicator judgment rules to generate performance stress testing tasks; based on the performance stress testing tasks, the tested version code is subjected to performance testing to obtain performance test result data. Through the above method, there is no need to perform performance testing in the production environment, nor is there any need to configure the test environment with the same resources as the production environment. It is only necessary to execute the generated performance stress testing tasks in a fixed test environment to obtain the performance test result data of the tested version code, thereby shielding the performance test's dependence on the tested environment configuration, thereby avoiding resource waste.

[0050] In some embodiments, based on the multi-level indicator judgment rule, the basic task is modeled and processed to generate a performance stress test task, including: executing the basic task to obtain the basic task execution result; the basic task execution result includes the first-level indicator, the second-level indicator and the third-level indicator, the first-level indicator includes the abnormality rate of each interface in the business system, the second-level indicator includes the overall resource utilization rate of the business system, and the third-level indicator includes the TPS value and the first time consumption in each thread group test result; judging whether the abnormality rate of each interface meets the first target requirement; if the abnormality rate of each interface meets the first target requirement, judging whether the overall resource utilization rate meets the second target requirement; if the overall resource utilization rate meets the second target requirement, If the rate does not meet the second target requirement, the basic task is adjusted to obtain the adjusted basic task, and the step of returning to execute the basic task and obtaining the execution result of the basic task is performed until the overall resource utilization rate meets the second target requirement; it is determined whether the TPS value and the first time consumption in the test result of each thread group meet the third target requirement; if the TPS value and the first time consumption in the test result of each thread group do not meet the third target requirement, the basic task is adjusted to obtain the adjusted basic task, and the step of returning to execute the basic task and obtaining the execution result of the basic task is performed until the TPS value and the first time consumption in the test result of each thread group meet the third target requirement, and a performance stress testing task is generated.

[0051] See also Figure 4 and Table 1, Figure 4 is a flow chart of the modeling processing provided in the embodiment of the present application, and Table 1 is an example of the multi-level indicator judgment rule provided in the embodiment of the present application.

[0052] Table 1

[0053] Specifically, the target requirements of the performance stress testing task are: the overall resource utilization cpu_limit of the business system needs to be between 20% and 80%, the first time consumption (that is, the time consumption for processing 90% of the tasks in the test results of each thread group) is less than 2000ms, and the abnormal rate is 0%.

[0054] like Figure 4 As shown in Table 1, after obtaining the basic task, the basic task is executed to obtain the basic task execution result. The basic task execution result includes primary indicators, secondary indicators and tertiary indicators. The primary indicator includes the exception rate of each interface in the business system, the secondary indicator includes the overall resource utilization rate of the business system, and the tertiary indicator includes the TPS value and the first time consumption in the test result of each thread group.

[0055] Furthermore, the abnormality rate of each interface in the aggregate report of the basic task execution result is traversed to determine whether the abnormality rate of each interface meets the first target requirement, that is, to determine whether the abnormality rate of each interface is 0%.

[0056] If the abnormal rate of any interface is not 0%, the first target requirement is not met. At this time, the secondary indicator judgment should be suspended, the model processing should be exited, and the abnormal request aggregation report information and the result tree information of the request abnormality should be displayed to facilitate manual troubleshooting and error reporting.

[0057] If the abnormal rate of each interface meets the first target requirement, that is, the abnormal rate of each interface is 0%, then enter the secondary indicator judgment: determine whether the overall resource utilization rate cpu_limit meets the second target requirement, that is, determine whether the overall resource utilization rate cpu_limit is in the preset threshold range (20%-80%).

[0058] If the overall resource utilization cpu_limit does not meet the second target requirement, that is, the overall resource utilization cpu_limit is not between 20% and 80%, the basic task is adjusted to adjust the pressure multiple value N (that is, TPS multiple value N), the initial timer setting number, and the initial concurrency setting value of the basic task, obtain the adjusted basic task, return to the step of executing the basic task, obtain the basic task execution result, and re-judge from the first-level indicator until the overall resource utilization cpu_limit meets the second target requirement.

[0059] Optionally, the base task may be adjusted up to 3 times.

[0060] Optionally, the pressure multiplier value may be gradually increased starting from 2.

[0061] If the overall resource usage cpu_limit meets the second target requirement, the third-level indicator judgment is entered: whether the TPS value and the first time consumption in the test result of each thread group meet the third target requirement.

[0062] If the TPS value and the first time consumption in the test results of each thread group do not meet the third target requirement, the basic task is adjusted to obtain the adjusted basic task, and the step of executing the basic task and obtaining the execution result of the basic task is returned. The judgment is restarted from the first-level indicator until the TPS value and the first time consumption in the test results of each thread group meet the third target requirement. In this case, the current basic task is used as the performance stress testing task.

[0063] After model processing, a performance stress testing task suitable for monitoring version performance can be generated.

[0064] Optionally, model processing also supports a simple version of direct task generation. Testers only need to enter the stress multiplier value N on the page and click Generate Task Directly to directly generate the performance stress test task corresponding to the basic task.

[0065] In some embodiments, the performance test result data includes system-level indicator data and business-level indicator data; based on the performance stress testing task, a performance test is performed on the tested version code, and after obtaining the performance test result data, it also includes: obtaining historical performance test result data of historical version codes; the historical performance test result data includes historical system-level indicator data and historical business-level indicator data; based on the business-level indicator data and the historical business-level indicator data, the business-level indicator analysis results of the tested version code are generated and displayed; based on the system-level indicator data and the historical system-level indicator data, the system-level indicator analysis results of the tested version code are generated and displayed; based on the business-level indicator analysis results and the system-level indicator analysis results, a performance trend curve is generated and displayed.

[0066] Please continue reading Figure 3 ,like Figure 3 As shown, the performance monitoring device of this embodiment can not only perform performance testing on the iterative version code in the test environment, but also perform comparative analysis based on the performance test results to timely discover performance problems, thereby achieving the purpose of reducing the performance risks introduced by iterative upgrades to the production environment.

[0067] Compared with traditional performance testing devices, the performance monitoring device of this embodiment has a model processing function, and can automatically generate performance stress testing tasks through model processing, without the need to manually formulate performance stress testing strategies and stress testing tasks. On this basis, the performance monitoring device can also connect to system monitoring (prometheus) and middleware monitoring, so as to perform intelligent analysis of performance test result data, and compare and display trends for key performance indicators.

[0068] See also Figure 5 , Figure 5 It is a flowchart of the intelligent comparative analysis provided in the embodiment of the present application.

[0069] In terms of performance test result analysis, traditional performance testing only focuses on the aggregated reports generated by the performance stress testing tasks and the system resource conditions when executing the performance stress testing tasks; while the performance monitoring device of this embodiment has an intelligent analysis function, and pays more attention to the comparison and trend changes of performance test result data between different versions, including comparative analysis of system-level indicator data and business-level indicator data, and the display of version performance trends, where the business level refers to the performance status of each service interface, and the system level refers to the system and middleware resource consumption.

[0070] Optionally, the system-level indicator data includes resource consumption and usage rates such as CPU consumption data, memory consumption data, and middleware (redis, database, ES, kafka, MQ) consumption data of the business system.

[0071] Business-level indicator data include performance parameters of each interface in the business system, TPS value of interface services (front-end and back-end), throughput, average response time, first response time, second response time, third response time, maximum response time, exception rate and number of requests, and other execution result indicators.

[0072] The first response time is the response time when processing 90% of the requests, the second response time is the response time when processing 95% of the requests, and the third response time is the response time when processing 99% of the requests.

[0073] The performance monitoring device has data storage capabilities, supports associating performance test result data with version iteration numbers, and saves the performance test result data executed by each version, including aggregate reports, system monitoring results, middleware monitoring results, and performance test reports generated by each version.

[0074] For example, assuming that version iteration numbers 200240801V1, 20240807V2, and 20240814V3 are respectively associated with performance stress test result data of three different versions, the performance monitoring device of this embodiment supports intelligent comparative analysis of the performance test result data of any two iteration versions.

[0075] Specifically, Figure 5 As shown, based on the performance stress testing task, the test version code is performance tested. After the performance test result data is obtained, the historical performance test result data of the historical version code can be obtained. The historical performance test result data includes historical system-level indicator data and historical business-level indicator data.

[0076] Furthermore, based on the business-level indicator data and historical business-level indicator data, the business-level indicator analysis results of the version code to be tested are generated and displayed; based on the system-level indicator data and historical system-level indicator data, the system-level indicator analysis results of the version code to be tested are generated and displayed.

[0077] Furthermore, based on the business-level indicator analysis results and the system-level indicator analysis results, a performance trend curve is generated and displayed.

[0078] In some embodiments, business-level indicator data include performance parameters of each interface in the business system: TPS value, throughput, average response time, first response time, second response time, third response time, maximum response time, exception rate and number of requests of the interface service, the first response time is the response time when processing 90% of the requests, the second response time is the response time when processing 95% of the requests, and the third response time is the response time when processing 99% of the requests. The business-level indicator analysis results include execution comparison results and exception comparison results; based on the business-level indicator data and historical business-level indicator data, the business-level indicator analysis results of the version code to be tested are generated and displayed, including: comparing and analyzing each performance parameter with the performance baseline, generating and displaying execution comparison results; calculating exception data based on the business-level indicator data and historical business-level indicator data, and generating and displaying exception comparison results.

[0079] The performance monitoring device has the function of setting the system performance baseline. Testers can set and modify the parameter values ​​in the performance baseline as needed; it also supports directly selecting the execution result of a certain performance stress test as the system performance baseline. The performance baseline contains business-level indicator parameters, such as the TPS value, throughput, average response time, first response time, second response time, third response time, maximum response time, abnormal rate, etc. of the interface service.

[0080] The performance monitoring device sets the system performance baseline and judges the business-level indicator data. It automatically analyzes and compares the result data of task execution (aggregation report, CPU monitoring data), displays the key indicators and abnormal data list, and can intuitively obtain the effective data of this performance stress test task without manual sorting, which is more conducive to judging whether the business-level indicator performance under the new version code (i.e. the version code to be tested) meets the system requirements.

[0081] Specifically, a historical version code (ie, a comparison version code) is first selected: assuming that the production environment adopts the historical version code V1, after executing the performance stress test task generated by the performance monitoring device, the historical performance test result data is saved as U1.

[0082] The version code to be tested in the test environment is the new version code V2. After the test environment is released and deployed, the performance stress test task generated by the performance monitoring device is also executed, and the performance test result data is saved as U2.

[0083] Furthermore, using the performance monitoring device, each performance parameter in U2 is compared and analyzed with the performance baseline, that is, the performance parameters of each interface in the aggregate report are compared with the performance baseline, and the change status of the data (such as increase / decrease in data) is highlighted, and the execution comparison results are generated and displayed.

[0084] Generally speaking, compared with the performance baseline, if each performance parameter of U2 does not deviate, it can be determined that the business-level indicators of the new version code V2 can meet production requirements; if any performance parameter of U2 deviates, it can be determined that the business-level indicators of the new version code V2 cannot meet production requirements.

[0085] Furthermore, abnormal data calculation is performed based on the business-level indicator data U2 and the historical business-level indicator data U1, and abnormal comparison results are generated and displayed.

[0086] It should be noted that the performance baseline in this embodiment is not fixed. Preferably, the performance test result data of the previous version (historical version) can be used as the performance baseline of the new version code V2 to form a comparison between the current version and the historical version to highlight the changes in the current version code.

[0087] Specifically, after generating a performance baseline based on the historical business-level indicator data in the historical performance test result data, abnormal data comparison is performed on each business-level indicator data of the new version code V2 and each historical business-level indicator data of the historical version code V1, and the abnormal data indicator is used as the abnormal comparison result and displayed to the tester to facilitate the tester to detect anomalies in a timely manner.

[0088] The calculation rules for abnormal data are as follows: (1) Benchmark value comparison: For business-level indicators such as average response time, first response time, second response time, third response time, maximum response time, and exception rate, compare them with the benchmark value of the performance baseline. If it is greater than the benchmark value, the indicator is determined to be an abnormal indicator. For business-level indicator throughput, compare it with the benchmark value of the performance baseline. If it is less than the benchmark value, the indicator is determined to be an abnormal indicator.

[0089] (2) For any business-level indicator, compare it with the corresponding indicator in the historical version code V1. If (new version indicator - historical version indicator) / historical version indicator >= 20% or = < - (20%), then the indicator is determined to be an abnormal indicator.

[0090] In some embodiments, the system-level indicator data includes CPU consumption data, memory consumption data, middleware consumption data and resource utilization rate; based on the system-level indicator data and historical system-level indicator data, the system-level indicator analysis results of the version code to be tested are generated and displayed, including: comparing and analyzing the system-level indicator data and the historical system-level indicator data to determine the resource consumption difference between the version code to be tested and the historical version code; based on the resource consumption difference, the system-level indicator analysis results of the version code to be tested are generated and displayed.

[0091] Similarly, by comparing with the selected historical version, that is, comparing the system and middleware resource consumption of different versions of code under the same performance stress testing task, the system-level performance differences of different versions can be directly observed, and it is easy to determine how much the new version of the code consumes more / less system and middleware resources than the current version, and whether there is a system-level performance risk.

[0092] Specifically, by comparing and analyzing the system-level indicator data in the performance test result data U2 and the historical system-level indicator data in the historical performance test result data U1, the resource consumption difference between the version code to be tested and the historical version code can be determined; based on the resource consumption difference, the system-level indicator analysis results of the version code to be tested are generated and displayed.

[0093] Generally, U1 and U2 are compared. Since the historical version code V1 runs in the production environment to meet the actual needs of users, when the performance data comparison results of each interface in U2 (including the front-end service interface and the back-end service interface) are better than those of U1, it can be judged that the new version code V2 can support actual production use after its release; when the performance data comparison results of U2 are worse than those of U1, it can be judged that the new version code V2 has performance risks, and performance tuning is required for the interface services with poor comparison results.

[0094] Among them, the comparison between U1 and U2 mainly focuses on whether the system-level indicators meet the conditions.

[0095] The performance monitoring device of this embodiment can be connected to the system monitoring platform and the middleware monitoring platform. It can not only display the system resource status during the performance test, but also digitally compare and analyze the data results with the selected comparison version, so as to intuitively feedback that the system resource consumption rate during the performance test execution under the current version code is increased / decreased by X% compared with the comparison version; at the same time, the performance monitoring device of this embodiment can also display the trend changes of system performance in each version iteration, which helps testers observe changes in system performance.

[0096] The performance testing method provided in the embodiment of the present application can conveniently and normally carry out performance testing on each iterative version in an agile iteration mode; in addition, through intelligent analysis, performance comparison can be supported. By comparing key performance parameter indicators and abnormal data display and system monitoring comparison, it is convenient for testers to quickly grasp the performance difference between the current version and the production version, and it is easy to make a clear judgment on whether the performance of the new version code can support user needs.

[0097] In some embodiments, interface service information is obtained, and basic tasks are generated based on the interface service information, including: obtaining interface service information through an interface platform and a log platform; calculating an initial TPS value, an initial timer setting number, and an initial concurrent number setting value based on the interface service information; and generating basic tasks based on the initial TPS value, the initial timer setting number, and the initial concurrent number setting value.

[0098] In summary, compared with the prior art, the performance testing method provided in the embodiments of the present application has at least the following advantages: (1) The model processing function provided in the embodiment of the present application can obtain data information such as service interfaces and business logs of the production environment, and automatically generate performance stress testing tasks containing all service interfaces. If the system has major adjustments, it only needs to re-acquire the data information once and regenerate the tasks, without the need for manual preset. There is no need to manually preset performance test plans for each business function module of the system, which avoids the possibility that performance stress testing tasks need to be adjusted or added when major functional adjustments occur in the business system, thereby effectively reducing the workload.

[0099] (2) The performance testing method provided in the embodiment of the present application not only supports the setting of system performance baselines and the comparison of execution results (i.e., business-level performance) between versions, but also adds the comparison of system-level performance, such as the performance parameters of the system CPU, memory, and middleware resources, thereby avoiding the possibility of inaccurate comparison results caused by only comparing business-level indicators, and effectively avoiding security risks.

[0100] The present application also provides a performance testing device. Figure 6 , Figure 6 6 is a schematic diagram of the structure of the performance testing device provided in the embodiment of the present application. In the embodiment of the present application, the performance testing device includes a publishing module 610, a basic task generating module 620, a stress testing task generating module 630 and a testing module 640.

[0101] The publishing module 610 is used to publish the code of the version to be tested of the business system to the test environment.

[0102] The basic task generation module 620 is used to obtain interface service information and generate basic tasks based on the interface service information.

[0103] The stress testing task generation module 630 is used to model the basic tasks based on the multi-level indicator judgment rules and generate performance stress testing tasks.

[0104] The testing module 640 is used to perform performance testing on the code of the tested version based on the performance stress testing task and obtain performance testing result data.

[0105] In some embodiments, the stress testing task generation module 630 is used to execute the basic task and obtain the basic task execution result; the basic task execution result includes the primary index, the secondary index and the tertiary index, the primary index includes the abnormality rate of each interface in the business system, the secondary index includes the overall resource utilization rate of the business system, and the tertiary index includes the TPS value and the first time consumption in each thread group test result; it is determined whether the abnormality rate of each interface meets the first target requirement; if the abnormality rate of each interface meets the first target requirement, it is determined whether the overall resource utilization rate meets the second target requirement; if the overall resource utilization rate does not meet the second target requirement, Then adjust the basic task to obtain the adjusted basic task, return to the step of executing the basic task and obtaining the execution result of the basic task, until the overall resource utilization rate meets the second target requirement; determine whether the TPS value and the first time consumption in the test result of each thread group meet the third target requirement; if the TPS value and the first time consumption in the test result of each thread group do not meet the third target requirement, adjust the basic task to obtain the adjusted basic task, return to the step of executing the basic task and obtaining the execution result of the basic task, until the TPS value and the first time consumption in the test result of each thread group meet the third target requirement, and generate a performance stress testing task.

[0106] In some embodiments, the performance test result data includes system-level indicator data and business-level indicator data.

[0107] Test module 640 is used to obtain historical performance test result data of historical version codes; the historical performance test result data includes historical system-level indicator data and historical business-level indicator data; based on the business-level indicator data and the historical business-level indicator data, the business-level indicator analysis results of the version code to be tested are generated and displayed; based on the system-level indicator data and the historical system-level indicator data, the system-level indicator analysis results of the version code to be tested are generated and displayed; based on the business-level indicator analysis results and the system-level indicator analysis results, a performance trend curve is generated and displayed.

[0108] In some embodiments, business-level indicator data include performance parameters of each interface in the business system: TPS value, throughput, average response time, first response time, second response time, third response time, maximum response time, exception rate and number of requests of the interface service. The first response time is the response time when processing 90% of the requests, the second response time is the response time when processing 95% of the requests, and the third response time is the response time when processing 99% of the requests. The business-level indicator analysis results include execution comparison results and exception comparison results.

[0109] The test module 640 is used to compare and analyze each performance parameter with the performance baseline, generate and display the execution comparison results; calculate abnormal data based on business-level indicator data and historical business-level indicator data, and generate and display abnormal comparison results.

[0110] In some embodiments, the system-level metric data includes CPU consumption data, memory consumption data, middleware consumption data, and resource usage.

[0111] The test module 640 is used to compare and analyze the system-level indicator data with the historical system-level indicator data to determine the resource consumption difference between the version code to be tested and the historical version code; based on the resource consumption difference, generate and display the system-level indicator analysis results of the version code to be tested.

[0112] In some embodiments, the basic task generation module 620 is used to obtain interface service information through the interface platform and the log platform; based on the interface service information, calculate the initial TPS value, the initial timer setting number and the initial concurrent number setting value; based on the initial TPS value, the initial timer setting number and the initial concurrent number setting value, generate a basic task.

[0113] The present application also provides an electronic device, Figure 7 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the performance testing method.

[0114] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0115] An embodiment of the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the performance testing methods provided by the above-mentioned methods.

[0116] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the performance testing methods provided by the above methods.

[0117] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A performance testing method, characterized in that: include: Release the code of the business system to be tested to the test environment; Acquire interface service information, and generate basic tasks based on the interface service information; Based on the multi-level indicator judgment rules, the basic tasks are modeled to generate performance stress testing tasks; Based on the performance stress testing task, a performance test is performed on the code of the version to be tested to obtain performance test result data.

2. The performance testing method according to claim 1, characterized in that: The basic tasks are modeled based on the multi-level indicator judgment rules to generate performance stress testing tasks, including: Execute the basic task to obtain the basic task execution result; the basic task execution result includes a primary indicator, a secondary indicator and a tertiary indicator, the primary indicator includes the abnormality rate of each interface in the business system, the secondary indicator includes the overall resource utilization rate of the business system, and the tertiary indicator includes the TPS value and the first time consumption in each thread group test result; Determining whether the abnormality rate of each of the interfaces meets the first target requirement; If the abnormal rate of each of the interfaces meets the first target requirement, then determining whether the overall resource utilization rate meets the second target requirement; If the overall resource utilization rate does not meet the second target requirement, the basic task is adjusted to obtain the adjusted basic task, and the process returns to the step of executing the basic task and obtaining the basic task execution result until the overall resource utilization rate meets the second target requirement; Determine whether the TPS value and the first time consumption in each of the thread group test results meet a third target requirement; If the TPS value and the first time consumption in each of the thread group test results do not meet the third target requirement, the basic task is adjusted to obtain the adjusted basic task, and the step of executing the basic task and obtaining the basic task execution result is returned until the TPS value and the first time consumption in each of the thread group test results meet the third target requirement, and a performance stress testing task is generated.

3. The performance testing method according to claim 1, characterized in that: The performance test result data includes system-level indicator data and business-level indicator data; After the performance test is performed on the code of the version to be tested based on the performance stress test task and the performance test result data is obtained, the method further includes: Obtain historical performance test result data of historical version codes; the historical performance test result data includes historical system-level indicator data and historical business-level indicator data; Based on the business-level indicator data and the historical business-level indicator data, generate and display the business-level indicator analysis results of the version code to be tested; Based on the system-level indicator data and the historical system-level indicator data, generate and display the system-level indicator analysis results of the version code to be tested; Based on the business-level indicator analysis results and the system-level indicator analysis results, a performance trend curve is generated and displayed.

4. The performance testing method according to claim 3, characterized in that: The business-level indicator data includes the performance parameters of each interface in the business system, the TPS value of the interface service, the throughput, the average response time, the first response time, the second response time, the third response time, the maximum response time, the exception rate and the number of requests. The first response time is the response time when processing 90% of the requests, the second response time is the response time when processing 95% of the requests, and the third response time is the response time when processing 99% of the requests. The business-level indicator analysis results include execution comparison results and exception comparison results; The generating and displaying the business-level indicator analysis result of the version code to be tested based on the business-level indicator data and the historical business-level indicator data includes: Compare and analyze each of the performance parameters with the performance baseline, and generate and display the execution comparison result; Abnormal data calculation is performed based on the business-level indicator data and the historical business-level indicator data, and the abnormal comparison result is generated and displayed.

5. The performance testing method according to claim 3, characterized in that: The system-level indicator data includes CPU consumption data, memory consumption data, middleware consumption data and resource utilization rate; The generating and displaying the system-level indicator analysis result of the version code to be tested based on the system-level indicator data and the historical system-level indicator data includes: Comparing and analyzing the system-level indicator data with the historical system-level indicator data to determine the resource consumption difference between the version code to be tested and the historical version code; Based on the resource consumption difference, a system-level indicator analysis result of the version code to be tested is generated and displayed.

6. The performance testing method according to claim 1, characterized in that: The obtaining of the interface service information and generating a basic task based on the interface service information includes: Obtain interface service information through the interface platform and log platform; Based on the interface service information, calculate the initial TPS value, the initial timer setting number and the initial concurrent number setting value; The basic task is generated based on the initial TPS value, the initial timer setting number and the initial concurrency setting value.

7. A performance testing device, characterized in that: include: The release module is used to release the code of the business system to be tested to the test environment; A basic task generation module, used to obtain interface service information and generate basic tasks based on the interface service information; A stress testing task generation module is used to model the basic tasks based on multi-level indicator judgment rules to generate performance stress testing tasks; The testing module is used to perform a performance test on the code of the version to be tested based on the performance stress testing task to obtain performance test result data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the performance testing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the performance testing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the performance testing method according to any one of claims 1 to 6 is implemented.

Images