Capacity calculation method and device based on full-link pressure test

Abstract

The invention discloses a capacity calculation method and device based on a full-link pressure test. The method comprises the following steps: analyzing according to a transaction service scene to obtain a transaction link; the transaction business scene is used for describing a specific business process and a system interaction process experienced from initiation, processing to completion of the transaction; building a test environment according to the transaction link, and deploying a test transaction model; based on the test environment, the transaction link and the test transaction model, initiating an end-to-end transaction test, monitoring a test execution process, and obtaining monitoring data; determining a mapping relationship between the resource utilization rate and throughput capacity of each subsystem according to the monitoring data, and calculating the throughput capacity of each subsystem according to the mapping relationship and the available resource utilization rate of each subsystem in the production environment; the available amount of the resource utilization rate represents the amount of resources meeting the safe operation of the subsystem. According to the invention, the capacity of each physical subsystem on the transaction link can be intelligently estimated, and the capacity estimation accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a capacity estimation method and apparatus based on end-to-end stress testing. Background Technology

[0002] As the operational unit responsible for ensuring safe production, data centers are accounted for in guaranteeing the stable operation of critical business scenarios. Production system capacity estimation is a crucial aspect of data center operation and management, its core significance stemming from the need for refined operational resource planning. Accurate capacity estimation supports production resource planning decisions and is an important basis for balancing resource input and output efficiency.

[0003] Commonly used methods for estimating system capacity each have their own characteristics but also limitations:

[0004] The production history data analysis method is only applicable to systems already in production. Because it relies entirely on production data, and stable production environments rarely sample data near resource bottlenecks, the data sampling points are mostly homogeneous data with low resource consumption. It cannot flexibly generate sampling data according to demand. Although production stress testing exists, it generally requires data staining and cleaning, resulting in high preparation and cleanup workloads and increasing the risk factors for safe production.

[0005] Stress testing in test environments typically focuses on single-system testing, primarily used to identify bottleneck deployment units within a single system. However, it cannot uncover potential performance issues caused by interdependencies between systems in the end-to-end transaction chain, and the accuracy of capacity estimation for each system is relatively poor. Summary of the Invention

[0006] This invention provides a capacity estimation method based on end-to-end stress testing to intelligently predict the capacity of each physical subsystem on the transaction link, thereby improving the accuracy of capacity prediction. The method includes:

[0007] The transaction chain is derived from the analysis of the transaction business scenario; the transaction chain is the path formed by multiple subsystems through which a transaction goes from initiation to completion; the transaction business scenario is used to describe the specific business processes and system interactions that a transaction undergoes from initiation, processing to completion.

[0008] A test environment is built based on the transaction chain, and a test transaction model is deployed. The test environment includes test servers for the subsystems in the corresponding transaction chain. The test transaction model is used to implement multiple transactions.

[0009] Based on the test environment, transaction chain, and test transaction model, initiate end-to-end transaction tests, monitor the test execution process, and obtain monitoring data; the monitoring data includes the resource utilization and throughput capacity of each subsystem.

[0010] The mapping relationship between resource utilization and throughput capacity of each subsystem is determined based on monitoring data. The throughput capacity of each subsystem is calculated based on the mapping relationship and the available resource utilization of each subsystem in the production environment. The available resource utilization represents the amount of resources required to ensure the safe operation of the subsystem.

[0011] This invention also provides a capacity estimation device based on end-to-end stress testing, used to intelligently estimate the capacity of each physical subsystem on the transaction link, thereby improving the accuracy of capacity estimation. The device includes:

[0012] The transaction link analysis module is used to analyze and obtain the transaction link based on the transaction business scenario. The transaction link is the path formed by multiple subsystems through which a transaction goes from initiation to completion. The transaction business scenario is used to describe the specific business process and system interaction process that a transaction goes through from initiation, processing to completion.

[0013] The test preparation module is used to build a test environment and deploy test transaction models based on the transaction chain. The test environment includes test servers for the subsystems in the corresponding transaction chain. The test transaction models are used to implement multiple transactions.

[0014] The test monitoring module is used to initiate end-to-end transaction tests based on the test environment, transaction chain, and test transaction model, monitor the test execution process, and obtain monitoring data; the monitoring data includes the resource utilization and throughput capacity of each subsystem.

[0015] The capacity estimation module is used to determine the mapping relationship between resource utilization and throughput capacity of each subsystem based on monitoring data, and to estimate the throughput capacity of each subsystem based on the mapping relationship and the available resource utilization of each subsystem in the production environment; the available resource utilization represents the amount of resources that can meet the safe operation of the subsystem.

[0016] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described capacity estimation method based on end-to-end stress testing.

[0017] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described capacity estimation method based on end-to-end stress testing.

[0018] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described capacity estimation method based on end-to-end stress testing.

[0019] This invention analyzes the transaction chain, builds a test environment, deploys a test transaction model, initiates end-to-end transaction tests, monitors the test execution process, obtains monitoring data, and determines the mapping relationship between resource utilization and throughput capacity of each subsystem based on the monitoring data. Based on the mapping relationship and the available resource utilization of each subsystem in the production environment, the throughput capacity of each subsystem is calculated. This achieves an automated, integrated, and intelligent solution from test environment construction, test scenario execution, to system capacity calculation. It can intelligently predict the capacity of each physical subsystem on the transaction chain, and improves the accuracy of capacity prediction by determining the mapping relationship between resource utilization and throughput capacity of each subsystem. This invention identifies performance bottleneck systems on the transaction chain in advance, discovers resource redundancy or insufficiency issues, provides data support for security risk management, and effectively ensures the continuous and stable operation of the production environment. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0021] Figure 1 This is a flowchart illustrating the capacity estimation method based on end-to-end stress testing in an embodiment of the present invention.

[0022] Figure 2 This is a specific example diagram of the capacity estimation method based on end-to-end stress testing in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of transaction link analysis in an embodiment of the present invention;

[0024] Figure 4 This is a trend chart of CPU utilization and TPS tested in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of a capacity estimation device based on end-to-end stress testing in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0027] To facilitate a clear description of the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order.

[0028] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0029] Existing capacity estimation methods have the following drawbacks:

[0030] 1) The method based on production history data analysis is only applicable to systems that are already in production. This method cannot be used to estimate capacity for newly built systems.

[0031] 2) Even for systems already in production, capacity estimation requirements are usually derived from a business promotion scenario that has never occurred in the past. The business scenarios in historical production data are very likely to be inconsistent with future business scenarios, which may lead to inaccuracies in the estimated capacity due to differences in the reference basis.

[0032] 3) The disadvantages of production pressure testing manufacturing data are obvious. In addition to increasing the amount of preparation and finishing work, the most important thing is that it increases the safety risk factors in production.

[0033] 4) Single-system stress testing lacks end-to-end full-link stress testing methods and constraints. It can only discover resource configuration problems within the system, but cannot discover whether the resource configuration between systems is reasonable.

[0034] To address at least one of the aforementioned shortcomings, embodiments of the present invention propose a capacity estimation method and apparatus for end-to-end stress testing. Figure 1 This is a flowchart illustrating the capacity estimation method based on end-to-end stress testing in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0035] Step 101: Analyze the transaction business scenario to obtain the transaction chain; the transaction chain is the path formed by multiple subsystems through which a transaction goes from initiation to completion; the transaction business scenario is used to describe the specific business process and system interaction process that a transaction goes through from initiation, processing to completion.

[0036] Step 102: Build a test environment and deploy a test transaction model based on the transaction chain; the test environment includes test servers for the subsystems in the corresponding transaction chain; the test transaction model is used to implement multiple transactions;

[0037] Step 103: Based on the test environment, transaction chain, and test transaction model, initiate an end-to-end transaction test, monitor the test execution process, and obtain monitoring data; the monitoring data includes the resource utilization and throughput capacity of each subsystem.

[0038] Step 104: Determine the mapping relationship between resource utilization and throughput capacity of each subsystem based on the monitoring data. Calculate the throughput capacity of each subsystem based on the mapping relationship and the available resource utilization of each subsystem in the production environment. The available resource utilization represents the amount of resources required to ensure the safe operation of the subsystem.

[0039] The capacity estimation method based on end-to-end stress testing in this embodiment of the invention is described in detail below.

[0040] Figure 2 This is a specific example diagram of the capacity estimation method based on end-to-end stress testing in an embodiment of the present invention. (Refer to...) Figure 2 This document demonstrates the implementation process of a capacity estimation method based on end-to-end stress testing. It utilizes a test resource management platform to automatically build and deliver test servers based on test resource requests. A non-functional automated testing platform initiates end-to-end test stress and compensation background stress in a tiered manner. The capacity management platform retrieves test reports from the non-functional automated testing platform and test environment information from the test resource management platform. Based on the capacity estimation model, it intelligently estimates the capacity of each physical subsystem in the transaction chain, identifies resource mismatches between systems in the chain, and pushes alerts to mobile terminals via instant messaging tools or emails, indicating potential capacity risks to operations managers. Specifically, the test resource management platform supports resource application and approval for the test environment and automatically builds and delivers servers. The non-functional automated testing platform is an automated performance stress testing tool platform that creates test tasks, establishes test scenarios, executes test scenarios, monitors test execution, and generates test reports. The capacity management platform's main functions include production resource management, capacity analysis and early warning, retrieving test reports and test scenario configuration information, performing capacity analysis and early warning based on the capacity estimation model, and notifying users of the capacity analysis results.

[0041] I. Analyze the transaction chain in end-to-end business scenarios.

[0042] In step 101, the transaction chain is obtained based on the transaction business scenario analysis; the transaction chain is the path formed by multiple subsystems through which the transaction goes from initiation to completion; the transaction business scenario is used to describe the specific business processes and system interaction processes that the transaction goes through from initiation, processing to completion.

[0043] Figure 3 This is a schematic diagram of transaction link analysis in an embodiment of the present invention, for reference. Figure 3A key business scenario during the 618 shopping festival involves systems A, B, C, and D. The transaction originates from front-end system A, which forwards the request to service provider system B. Service provider system B, based on the payment method used by the user, forwards the request to different accounting systems C or D. By analyzing the transaction scenario, the transaction chain is obtained, and the physical subsystems involved in the transaction chain are identified.

[0044] The transaction chain includes multiple subsystems, including the main testing system and related systems. The main testing system is the core target of the test, while the related systems are other subsystems involved in the transaction chain besides the main testing system, such as databases and caching systems.

[0045] During implementation, the decision can be made based on one or any combination of factors, such as software version changes, the importance of the transaction, and identified risks.

[0046] (1) In the transaction chain, the ABCD system has undergone a version change in this production, while the versions of ACD have not changed;

[0047] (2) If there are changes in all of A, B, C, and D in the transaction chain, the main test system shall be determined according to the degree of relevance to the business scenario;

[0048] (3) If System B discovers a risk or problem, it needs to conduct a special test on the risk or problem.

[0049] In one embodiment, the transaction data includes a transaction identifier, transaction time, and subsystem information;

[0050] The transaction chain can be obtained by analyzing transaction data, and may also include: reconstructing the transaction chain based on transaction identifiers, transaction times, and subsystem information to achieve accurate tracking of the entire chain.

[0051] II. Set up an end-to-end testing environment.

[0052] In step 102, a test environment is set up according to the transaction chain. The test environment includes test servers for the subsystems in the corresponding transaction chain.

[0053] Based on the transaction business scenario and the transaction functions that each subsystem can achieve, the transaction chain is analyzed.

[0054] End-to-end stress testing involves multiple physical subsystems. Building a test environment proportional to production resource allocation would require significant resources, leading to substantial resource supply difficulties. Therefore, without affecting test results, the test environment topology should be consistent with the production environment, and resources should be adjusted accordingly.

[0055] In one embodiment, building a test environment based on the transaction chain may include:

[0056] Based on the resource usage of each subsystem in the production environment, the resources for deploying the test server are calculated and adjusted according to a preset ratio.

[0057] For example, bottleneck deployment units of each physical subsystem can be calculated at a certain ratio, but to reduce errors in subsequent capacity estimation, the ratio should not be too small. For instance, the ratio of minimum testing to production resources can be controlled at 1 / 10. Other non-bottleneck deployment units should apply for resources according to the principle of minimum necessity.

[0058] The transaction chain consists of multiple systems A, B, C, and D. A certain system, such as B, includes multiple deployment units, among which the deployment unit with the highest resource utilization rate is the bottleneck deployment unit.

[0059] In one embodiment, building a test environment based on the transaction chain may include:

[0060] Based on the resource usage of the main test system in the production environment, determine the resource conversion ratio of the test server corresponding to the main test system;

[0061] Based on the resource conversion ratio of the test server corresponding to the main test system, determine the resource conversion ratio of other test servers.

[0062] For example, the test-to-production conversion ratio of each physical subsystem can be kept consistent with the conversion ratio of the main test system, making it convenient to directly obtain from the test results which system in the transaction chain is most likely to reach the resource bottleneck first.

[0063] During implementation, first determine that the resource conversion ratio of the main test system is greater than or equal to 1 / 10, such as 1 / 5. Then, calculate the test resource requirements by referring to the resource conversion ratio of other subsystems at 1 / 5.

[0064] In this embodiment of the invention, the relationships between the transaction link, subsystem, deployment unit, number of servers, and configuration of a single server are as follows:

[0065] (1) A transaction chain includes multiple subsystems (one of which is designated as the main testing system, and the others are related systems);

[0066] (2) A subsystem consists of multiple deployment units;

[0067] (3) A deployment unit consists of multiple servers;

[0068] (4) Each server has different configurations, such as processor and memory.

[0069] When setting up a test environment, if the single-system stress test has not been verified, the server can be scaled vertically in a linear manner. It is recommended that the configuration of a single server be consistent with that of the production system to reduce capacity estimation errors caused by differences in the configuration of a single server.

[0070] In one embodiment, the test server is deployed at the same physical location as the corresponding subsystem.

[0071] For example, the deployment areas of multiple physical subsystems should be consistent with the production environment. This is because production experiences network latency due to cross-regional deployments (affecting transaction response time). Therefore, test environment requests should refer to production scenarios when requesting cross-regional resources to simulate real-world production conditions as closely as possible.

[0072] In addition, when building the test environment, it is important to ensure the consistency of technical parameters between the production and test environments. These parameters include key operating system parameters, key database parameters, key parameters of various middleware, software runtime configuration parameters, JVM configuration items, and other application parameters.

[0073] 3. Build test transaction models based on business scenarios.

[0074] In step 102, a test trading model is deployed to implement multiple transactions.

[0075] For example, determine the algorithmic functional modules that implement each transaction, as well as the transactions that each subsystem needs to implement, and then deploy them.

[0076] In one embodiment, deploying a test trading model may include:

[0077] Retrieve the transaction list of each subsystem; each column in the transaction list includes the transaction name, transaction percentage, and transaction path, etc. The transaction list is sorted from largest to smallest transaction percentage;

[0078] The terms for setting percentages before retrieving the transaction list form the test transaction models for each subsystem;

[0079] Deploy the test transaction models of each subsystem to the corresponding test server of each subsystem.

[0080] Table 1 shows the transaction list for the online transaction business scenario of physical subsystem A, and Table 2 shows the transaction list for the online transaction business scenario of physical subsystem C.

[0081] Table 1

[0082]

[0083] Table 2

[0084]

[0085] The business scenarios of each physical subsystem along the transaction chain are analyzed sequentially, and typical online transaction business scenarios are summarized (generally taking >=80% of the top transactions). The content includes the transaction list, transaction percentage, transaction path (including branches), peak TPS, etc. The online transaction business scenarios of non-production systems are estimated based on business needs. For production systems, if the capacity estimation scenarios need to be consistent with the production situation, references can be obtained from production monitoring.

[0086] Once the business scenario is clearly defined, the test transaction model can be proportionally increased to a total of 100% based on the business scenario.

[0087] IV. Initiating end-to-end pressure and background compensation pressure.

[0088] In step 103, based on the test environment, transaction link, and test transaction model, an end-to-end transaction test is initiated, the test execution process is monitored, and monitoring data is obtained; the monitoring data includes the resource utilization and throughput capacity of each subsystem.

[0089] In the test environment, based on the transaction link and test transaction model, the real transaction process is fully simulated from the user initiation end to each backend subsystem to verify the system function, link connectivity, data consistency and performance, to ensure that the coordination of each link is correct and to discover integration and logic defects in advance.

[0090] In one embodiment, initiating an end-to-end transaction test includes:

[0091] Based on the test environment, transaction link, and test transaction model, pressure is applied in multiple gradient increments; the gradient increments include: the difference in resource utilization rates between two adjacent gradients reaches a specified percentage or more of the resource utilization rate of the previous gradient, or the difference in throughput capacity between two adjacent gradients reaches a specified percentage or more of the throughput capacity of the previous gradient.

[0092] For example, based on the end-to-end test transaction model of the critical business assurance scenario under test, increase the background pressure of each physical subsystem, and incrementally design four or more gradients of test pressure (e.g., corresponding to CPU utilization of AP at 20%, 40%, 60%, and 80-100%). Generally, the second-to-last gradient is designed as the optimal gradient, for example, the optimal gradient is AP <= 60%, DB <= 80%, where AP is the application server and DB is the database server.

[0093] Apply pressure in an incremental gradient to ensure that there is a certain difference in TPS or CPU resource utilization between two adjacent gradients, such as a difference of more than 20%. Each gradient should run stably for a period of time, such as 10 minutes.

[0094] Referring to Table 4, physical subsystem C, in addition to receiving transaction requests C001 and C002 from initiator A, also receives transactions C003, C004, and C005 from other initiators. Even C001 and C002 may contain transactions from other initiators. Therefore, in the end-to-end stress testing scheme for a critical business scenario like the 618 shopping festival, it is necessary to consider that the actual calling relationships between various physical subsystems are a complex and interwoven network structure, and these derived transactions will inevitably affect capacity estimation. On the other hand, it is impossible to build an end-to-end environment for all other transactions. To resolve this contradiction, this embodiment of the invention proposes a method for end-to-end end-to-end test stress of the main scenario under test, combined with background compensation stress from derived transaction initiation, to simulate the real transaction pressure of each physical subsystem in production, thereby eliminating the impact of derived links on capacity estimation.

[0095] In one embodiment, initiating an end-to-end transaction test includes:

[0096] Monitor whether the resource utilization rate of each subsystem in the transaction chain reaches the preset threshold; the preset threshold is used to indicate that the system has reached the optimal utilization state.

[0097] For subsystems where resource utilization has not reached a preset threshold: increase background compensation pressure until resource utilization reaches the preset threshold; the background compensation pressure is issued according to the subsystem's test transaction model.

[0098] For example, during optimal gradient stress testing, it is checked whether the resource utilization of each physical subsystem on the end-to-end transaction link has reached its optimal state (e.g., AP <= 60%, DB <= 80%). If it is found that some physical subsystems have not reached the preset resource threshold, background compensation pressure needs to be increased until their resource utilization reaches the optimal level. Optimal resource utilization helps improve the accuracy of capacity estimation for each physical subsystem on the transaction link.

[0099] The transaction list and transaction percentage for the background test stress should refer to the test transaction model of the physical subsystem. After overlaying the end-to-end stress and the background compensation test stress, the transaction percentage of each physical subsystem should remain unchanged in the test scenario, and the test transaction model should not deviate.

[0100] Obtain monitoring data, such as CPU utilization and throughput capacity, such as system processing power (TPS).

[0101] During implementation, the system processing power (TPS), transaction response time, transaction success rate, and resource usage were observed and recorded for different gradient stress tests, as shown in Tables 3(1) and 3(2). Tables 3(1) and 3(2) show the monitoring data for each gradient. Table 4 shows the resource usage.

[0102] Table 3(1)

[0103]

[0104] Table 3(2)

[0105]

[0106] Table 4

[0107]

[0108] Finally, the test ends when the gradient pressure meets the test exit conditions, such as reaching a resource threshold, exceeding the processing time limit, TPS decrease, or other constraints.

[0109] V. Estimate the capacity of each physical subsystem on the transaction link based on the test data.

[0110] In step 105, the mapping relationship between resource utilization and throughput capacity of each subsystem is determined based on monitoring data. Based on the mapping relationship and the available resource utilization of each subsystem in the production environment, the throughput capacity of each subsystem is calculated. The available resource utilization represents the amount of resources required to ensure the safe operation of the subsystem.

[0111] As shown in Tables 3 and 4, the monitoring data can include the resource utilization and throughput capacity of each subsystem at each gradient.

[0112] Determining the mapping relationship between resource utilization and throughput capacity of each subsystem based on monitoring data can include:

[0113] By using the resource utilization and throughput capacity of each gradient to perform linear regression fitting, the mapping relationship between the resource utilization and throughput capacity of each subsystem is obtained.

[0114] Figure 4 The following is a trend chart of CPU utilization and TPS tested in an embodiment of the present invention, for reference. Figure 4 The data shows the trends of TPS and CPU resource utilization at various gradients. It can be found that the two have a significant linear relationship before the performance inflection point. Therefore, linear regression fitting can quantify the trend, simplify the complex relationship, and facilitate analysis, monitoring and decision-making.

[0115] In one embodiment, determining the mapping relationship between resource utilization and throughput capacity of each subsystem based on monitoring data may include: fitting a linear equation in two variables using the resource utilization and throughput capacity of each gradient to obtain the mapping relationship between resource utilization and throughput capacity of each subsystem.

[0116] For example, a linear equation in two variables can be fitted with the increase in throughput capacity as the independent variable and the increase in resource utilization rate as the dependent variable.

[0117] The calculation steps for estimating production capacity using a system of two linear equations in two variables are as follows:

[0118] 1) Assuming that the incremental TPS and incremental CPU utilization have a linear growth relationship, let x be the incremental TPS and y be the incremental CPU utilization. The linear fitting model is y = a × x + b.

[0119] 2) Based on the gradient stress test results, substitute the specific values ​​of incremental TPS and incremental CPU (such as the third gradient minus the second gradient, and the fourth gradient minus the second gradient), list the equations, solve the equations, and calculate the specific values ​​of coefficients a and b.

[0120] For example: .

[0121] The throughput capacity of each subsystem is calculated by using this linear equation in two variables and the available resource utilization of each subsystem in the production environment.

[0122] In one embodiment, the available resource utilization is the difference between the maximum resource utilization threshold and the resource utilization when the server is idle.

[0123] During implementation, the total incremental CPU utilization (y) available for production is calculated based on the production server resources. Since server resource utilization exists even under no-load conditions (e.g., CPU overhead from the OS and resident system services), this resource usage is not directly related to TPS. Let z be the CPU utilization when the server is idle. Assuming each server's resource utilization reaches a threshold of 60%, the theoretically optimal production scenario yields the total incremental CPU utilization = (60% - z) × (number of bottleneck deployment units × production / test resource configuration ratio per server). For example, a configuration ratio of 4:1 = (16 CPUs per server in production environment / 4 CPUs per server in test environment).

[0124] Substitute the y value (the sum of incremental utilization of production CPUs), the value of coefficient a and the value of coefficient b calculated in step 2) into the formula x=(yb) / a, and calculate the capacity x value of each physical subsystem in turn.

[0125] In one embodiment, after calculating the throughput capacity of each subsystem based on the mapping relationship and the available resource utilization of each subsystem in the production environment, the method further includes:

[0126] The ratio of the estimated throughput capacity of each subsystem to the throughput capacity of each subsystem in the production environment is calculated to obtain the ratio of each subsystem.

[0127] Based on the differences in the ratios of the various subsystems, identify the subsystems with resource bottlenecks and / or those with resource waste.

[0128] For example, the ratio of the calculated throughput capacity of each subsystem to the throughput capacity of each subsystem in the production environment is recorded as the capacity redundancy multiple of each physical subsystem. The capacity redundancy multiple of each physical subsystem on the transaction link = calculated TPS / production index TPS. The system with the smallest redundancy multiple is the bottleneck system in the end-to-end transaction link. A multiple greater than 5 times indicates potential resource waste, a multiple between [1,2] indicates insufficient redundancy, and a multiple between [0,1] indicates insufficient capacity. Refer to Table 5 for the full-link capacity analysis results of a key business scenario during the 618 promotion. The analysis results show that systems C and D have unreasonable resource configurations. System C is the bottleneck system of the entire link with insufficient redundancy, while system D has excessive redundancy, resulting in resource waste.

[0129] Table 5. Full-link capacity analysis results for a key business during the 618 promotion.

[0130]

[0131] In summary, this invention, relying on a test resource management platform, a non-functional automated testing platform, and a capacity management platform, provides an automated, integrated, and intelligent solution for end-to-end capacity estimation in critical business assurance scenarios. This solution encompasses test environment construction, test scenario execution, and system capacity estimation. By using automated capacity analysis devices to proactively identify performance bottlenecks in the transaction chain and detect resource redundancy or insufficiency, it provides data support for security risk mitigation and effectively ensures the continuous and stable operation of the production environment.

[0132] The embodiments of this invention clarify the basic principles to be followed in building an end-to-end full-link test environment and introduce a mechanism for background supplementary test pressure, thereby effectively avoiding the impact of other exchange-derived links on capacity estimation due to differences between production and test environments and links in non-test business guarantee scenarios.

[0133] This invention proposes a capacity estimation model based on a system of two linear equations: y = a × x + b. By substituting the specific values ​​of the incremental TPS and incremental CPUs between gradient stress tests, the values ​​of coefficients a and b are calculated. Finally, by substituting the total incremental CPUs usable in production, the system capacity TPS is estimated.

[0134] This invention also provides a capacity estimation device based on end-to-end stress testing, as described in the following embodiments. Since the principle by which this device solves the problem is similar to the capacity estimation method based on end-to-end stress testing, the implementation of this device can refer to the implementation of the capacity estimation method based on end-to-end stress testing; repeated details will not be elaborated further.

[0135] Figure 5 This is a schematic diagram of a capacity estimation device based on end-to-end stress testing in an embodiment of the present invention, as shown below. Figure 5 As shown, the device 500 includes:

[0136] The transaction link analysis module 501 is used to analyze and obtain the transaction link based on the transaction business scenario. The transaction link is the path formed by multiple subsystems through which a transaction goes from initiation to completion. The transaction business scenario is used to describe the specific business process and system interaction process that the transaction goes through from initiation, processing to completion.

[0137] Test preparation module 502 is used to build a test environment and deploy test transaction models according to the transaction chain; the test environment includes test servers for the subsystems in the corresponding transaction chain; the test transaction models are used to implement multiple transactions.

[0138] The test monitoring module 503 is used to initiate end-to-end transaction tests based on the test environment, transaction chain, and test transaction model, monitor the test execution process, and obtain monitoring data; the monitoring data includes the resource utilization and throughput capacity of each subsystem.

[0139] The capacity estimation module 504 is used to determine the mapping relationship between the resource utilization rate and throughput capacity of each subsystem based on monitoring data, and to estimate the throughput capacity of each subsystem based on the mapping relationship and the available amount of resource utilization rate of each subsystem in the production environment; the available amount of resource utilization rate represents the amount of resources that can meet the safe operation of the subsystem.

[0140] In one embodiment, the test preparation module 502 is specifically used for:

[0141] Based on the resource usage of each subsystem in the production environment, the resources for deploying the test server are calculated and adjusted according to a preset ratio.

[0142] In one embodiment, the transaction chain includes multiple subsystems, including a main testing system; the main testing system is determined based on one or any combination of factors such as software version changes, the importance of executing the transaction, and risk detection.

[0143] Test preparation module 502 is specifically used for:

[0144] Based on the resource usage of the main test system in the production environment, determine the resource conversion ratio of the test server corresponding to the main test system;

[0145] Based on the resource conversion ratio of the test server corresponding to the main test system, determine the resource conversion ratio of other test servers.

[0146] In one embodiment, the test server is deployed at the same physical location as the corresponding subsystem.

[0147] In one embodiment, the test preparation module 502 is specifically used for:

[0148] Retrieve the transaction list of each subsystem; each column in the transaction list includes the transaction name and transaction percentage, and the transaction list is sorted from largest to smallest transaction percentage.

[0149] The terms for setting percentages before retrieving the transaction list form the test transaction models for each subsystem;

[0150] Deploy the test transaction models of each subsystem to the corresponding test server of each subsystem.

[0151] In one embodiment, the test monitoring module 503 is specifically used for:

[0152] Based on the test environment, transaction link, and test transaction model, pressure is applied in multiple gradient increments; the gradient increments include: the difference in resource utilization rates between two adjacent gradients reaches a specified percentage or more of the resource utilization rate of the previous gradient, or the difference in throughput capacity between two adjacent gradients reaches a specified percentage or more of the throughput capacity of the previous gradient.

[0153] In one embodiment, the test monitoring module 503 is specifically used for:

[0154] Monitor whether the resource utilization rate of each subsystem in the transaction chain reaches the preset threshold;

[0155] For subsystems where resource utilization has not reached a preset threshold: increase background compensation pressure until resource utilization reaches the preset threshold; the background compensation pressure is issued according to the subsystem's test transaction model.

[0156] In one embodiment, the monitoring data includes resource utilization and throughput capacity of each subsystem at each gradient;

[0157] The capacity estimation module 504 is specifically used for:

[0158] By using the resource utilization and throughput capacity of each gradient to perform linear regression fitting, the mapping relationship between the resource utilization and throughput capacity of each subsystem is obtained.

[0159] In one embodiment, the capacity estimation module 504 is specifically used for:

[0160] By fitting a linear equation in two variables using the resource utilization and throughput capacity of each gradient, the mapping relationship between the resource utilization and throughput capacity of each subsystem is obtained.

[0161] In one embodiment, the available resource utilization is the difference between the maximum resource utilization threshold and the resource utilization when the server is idle.

[0162] In one embodiment, the device 500 may further include:

[0163] The redundancy judgment module is used by the capacity estimation module 504 to calculate the throughput capacity of each subsystem after estimating the throughput capacity of each subsystem based on the mapping relationship and the available resource utilization of each subsystem in the production environment, and then calculates the ratio of the estimated throughput capacity of each subsystem to the throughput capacity of each subsystem in the production environment to obtain the ratio value of each subsystem.

[0164] Based on the differences in the ratios of the various subsystems, identify the subsystems with resource bottlenecks and / or those with resource waste.

[0165] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described capacity estimation method based on end-to-end stress testing.

[0166] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described capacity estimation method based on end-to-end stress testing.

[0167] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described capacity estimation method based on end-to-end stress testing.

[0168] This invention analyzes the transaction chain, builds a test environment, deploys a test transaction model, initiates end-to-end transaction tests, monitors the test execution process, obtains monitoring data, and determines the mapping relationship between resource utilization and throughput capacity of each subsystem based on the monitoring data. Based on the mapping relationship and the available resource utilization of each subsystem in the production environment, the throughput capacity of each subsystem is calculated. This achieves an automated, integrated, and intelligent solution from test environment construction, test scenario execution, to system capacity calculation. It can intelligently predict the capacity of each physical subsystem on the transaction chain, and improves the accuracy of capacity prediction by determining the mapping relationship between resource utilization and throughput capacity of each subsystem. This invention identifies performance bottleneck systems on the transaction chain in advance, discovers resource redundancy or insufficiency issues, provides data support for security risk management, and effectively ensures the continuous and stable operation of the production environment.

[0169] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A capacity estimation method based on end-to-end stress testing, characterized in that, include: The transaction chain is derived from the analysis of the transaction business scenario; the transaction chain is the path formed by multiple subsystems through which a transaction goes from initiation to completion; the transaction business scenario is used to describe the specific business processes and system interactions that a transaction undergoes from initiation, processing to completion. A test environment is built based on the transaction chain, and a test transaction model is deployed. The test environment includes test servers for the corresponding subsystems in the transaction chain. The test trading model is used to implement multiple trades; Based on the test environment, transaction chain, and test transaction model, initiate end-to-end transaction tests, monitor the test execution process, and obtain monitoring data; the monitoring data includes the resource utilization and throughput capacity of each subsystem. The mapping relationship between resource utilization and throughput capacity of each subsystem is determined based on monitoring data. The throughput capacity of each subsystem is calculated based on the mapping relationship and the available resource utilization of each subsystem in the production environment. The available resource utilization rate represents the amount of resources required to ensure the safe operation of the subsystem.

2. The method as described in claim 1, characterized in that, The test environment was set up based on the transaction chain, including: Based on the resource usage of each subsystem in the production environment, the resources for deploying the test server are calculated and adjusted according to a preset ratio.

3. The method as described in claim 1, characterized in that, The transaction chain includes multiple subsystems, including a main testing system; the main testing system is determined based on one or any combination of factors such as software version changes, the importance of executing the transaction, and risk discovery. The test environment was set up based on the transaction chain, including: Based on the resource usage of the main test system in the production environment, determine the resource conversion ratio of the test server corresponding to the main test system; Based on the resource conversion ratio of the test server corresponding to the main test system, determine the resource conversion ratio of other test servers.

4. The method as described in claim 1, characterized in that, The actual physical location of the test server deployment is the same as the actual physical location of the corresponding subsystem.

5. The method as described in claim 1, characterized in that, Deploy and test the trading model, including: Retrieve the transaction list of each subsystem; each column in the transaction list includes the transaction name and transaction percentage, and the transaction list is sorted from largest to smallest transaction percentage. The terms for setting percentages before retrieving the transaction list form the test transaction models for each subsystem; Deploy the test transaction models of each subsystem to the corresponding test server of each subsystem.

6. The method as described in claim 1, characterized in that, Initiate end-to-end transaction testing, including: Based on the test environment, transaction link, and test transaction model, pressure is applied in multiple gradient increments; the gradient increments include: the difference in resource utilization rates between two adjacent gradients reaches a specified percentage or more of the resource utilization rate of the previous gradient, or the difference in throughput capacity between two adjacent gradients reaches a specified percentage or more of the throughput capacity of the previous gradient.

7. The method as described in claim 6, characterized in that, Initiate end-to-end transaction testing, including: Monitor whether the resource utilization rate of each subsystem in the transaction chain reaches the preset threshold; For subsystems where resource utilization has not reached a preset threshold: increase background compensation pressure until resource utilization reaches the preset threshold; the background compensation pressure is issued according to the subsystem's test transaction model.

8. The method as described in claim 6, characterized in that, The monitoring data includes the resource utilization and throughput capacity of each subsystem at each gradient; Based on monitoring data, the mapping relationship between resource utilization and throughput capacity of each subsystem is determined, including: By using the resource utilization and throughput capacity of each gradient to perform linear regression fitting, the mapping relationship between the resource utilization and throughput capacity of each subsystem is obtained.

9. The method as described in claim 6, characterized in that, Based on monitoring data, the mapping relationship between resource utilization and throughput capacity of each subsystem is determined, including: By fitting a linear equation in two variables using the resource utilization and throughput capacity of each gradient, the mapping relationship between the resource utilization and throughput capacity of each subsystem is obtained.

10. The method as described in claim 1, characterized in that, The available resource utilization rate is the difference between the maximum resource utilization rate threshold and the resource utilization when the server is idle.

11. The method as described in claim 1, characterized in that, After calculating the throughput capacity of each subsystem based on the mapping relationship and the available resource utilization of each subsystem in the production environment, the following steps are also included: The ratio of the estimated throughput capacity of each subsystem to the throughput capacity of each subsystem in the production environment is calculated to obtain the ratio of each subsystem. Based on the differences in the ratios of the various subsystems, identify the subsystems with resource bottlenecks and / or those with resource waste.

12. A capacity estimation device based on end-to-end stress testing, characterized in that, include: The transaction link analysis module is used to analyze and obtain the transaction link based on the transaction business scenario. The transaction link is the path formed by multiple subsystems through which a transaction goes from initiation to completion. The transaction business scenario is used to describe the specific business process and system interaction process that a transaction goes through from initiation, processing to completion. The test preparation module is used to build a test environment and deploy test transaction models based on the transaction chain; the test environment includes test servers for the subsystems in the corresponding transaction chain. The test trading model is used to implement multiple trades; The test monitoring module is used to initiate end-to-end transaction tests based on the test environment, transaction chain, and test transaction model, monitor the test execution process, and obtain monitoring data; the monitoring data includes the resource utilization and throughput capacity of each subsystem. The capacity estimation module is used to determine the mapping relationship between resource utilization and throughput capacity of each subsystem based on monitoring data, and to estimate the throughput capacity of each subsystem based on the mapping relationship and the available resource utilization of each subsystem in the production environment. The available resource utilization rate represents the amount of resources required to ensure the safe operation of the subsystem.

13. The apparatus as claimed in claim 12, characterized in that, The test preparation module is specifically used for: Retrieve the transaction list of each subsystem; each column in the transaction list includes the transaction name and transaction percentage, and the transaction list is sorted from largest to smallest transaction percentage. The terms for setting percentages before retrieving the transaction list form the test transaction models for each subsystem; Deploy the test transaction models of each subsystem to the corresponding test server of each subsystem.

14. The apparatus as claimed in claim 12, characterized in that, The test monitoring module is specifically used for: Based on the test environment, transaction link, and test transaction model, pressure is applied in multiple gradient increments; the gradient increments include: the difference in resource utilization rates between two adjacent gradients reaches a specified percentage or more of the resource utilization rate of the previous gradient, or the difference in throughput capacity between two adjacent gradients reaches a specified percentage or more of the throughput capacity of the previous gradient.

15. The apparatus as claimed in claim 14, characterized in that, The test monitoring module is specifically used for: Monitor whether the resource utilization rate of each subsystem in the transaction chain reaches the preset threshold; For subsystems where resource utilization has not reached the preset threshold: increase background compensation pressure until resource utilization reaches the preset threshold; The background compensation pressure is issued according to the test transaction model of the subsystem.

16. The apparatus as claimed in claim 14, characterized in that, The monitoring data includes the resource utilization and throughput capacity of each subsystem at each gradient; The capacity estimation module is specifically used for: By using the resource utilization and throughput capacity of each gradient to perform linear regression fitting, the mapping relationship between the resource utilization and throughput capacity of each subsystem is obtained.

17. The apparatus as claimed in claim 14, characterized in that, The capacity estimation module is specifically used for: By fitting a linear equation in two variables using the resource utilization and throughput capacity of each gradient, the mapping relationship between the resource utilization and throughput capacity of each subsystem is obtained.

18. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 11.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 11.

20. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 11.

Images