System performance evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides a system performance evaluation method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a reference performance index parameter of a new business system and component calling information of a plurality of business components in the new business system, the component calling information being used for indicating calling conditions of the corresponding business components in a preset transaction test state; according to the benchmark performance index parameters and the component calling information, a preset system evaluation model is adopted, and reference performance index parameters of the multiple service components are calculated; the reference performance index parameters of the plurality of service components are used for providing reference performance indexes for updating of the plurality of service components. According to the invention, the reference performance index of each service component in the new service system can be accurately evaluated, so that updating and optimization of the service components are supported.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a system performance evaluation method, device, electronic device and storage medium. Background Art

[0002] In the banking industry, as business needs develop, new business systems need to be developed to meet new business needs.

[0003] During the development of a new business system, it is necessary to evaluate the throughput and response time that each business component in the new business system can support in a transaction.

[0004] In the existing technology, the newly built business system is often evaluated with reference to the production system that has been put into operation. When there is no production system for reference, the evaluation is entirely based on personal experience and cannot be accurate. Summary of the invention

[0005] The purpose of this application is to provide a system performance evaluation method, device, electronic device and storage medium to address the deficiencies in the above-mentioned prior art, so as to accurately evaluate the reference performance indicators of each business component in a newly built business system, thereby supporting the update and optimization of business components.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a system performance evaluation method, the method comprising:

[0008] Obtaining a benchmark performance indicator parameter of a newly created business system and component call information of multiple business components in the newly created business system, wherein the component call information is used to indicate a call status of a corresponding business component in a preset transaction test state;

[0009] According to the benchmark performance indicator parameters and the component call information, a preset system evaluation model is used to calculate reference performance indicator parameters of the multiple business components; the reference performance indicator parameters of the multiple business components are used to provide reference performance indicators for updating the multiple business components.

[0010] Optionally, the benchmark performance indicator parameters include: benchmark parameters of at least two performance indicators; the preset system evaluation model includes: evaluation algorithms of the at least two performance indicators; the reference performance indicator parameters include: reference values ​​for the at least two performance indicators;

[0011] The step of calculating reference performance indicator parameters of the plurality of business components using a preset system evaluation model according to the benchmark performance indicator parameters and the component call information includes:

[0012] According to the benchmark parameters of the at least two performance indicators and the component calling information of each business component, the evaluation algorithms of the at least two performance indicators are respectively adopted to calculate the reference value of each business component for the at least two performance indicators.

[0013] Optionally, the benchmark parameters of the at least two performance indicators include: a benchmark parameter of a throughput indicator, the evaluation algorithms of the at least two performance indicators include: a first evaluation algorithm, and the component call information of each business component includes: a call count;

[0014] The step of calculating the reference performance indicator parameters of each business component by using the evaluation algorithms of the at least two performance indicators respectively according to the benchmark parameters of the at least two performance indicators and the component call information of each business component includes:

[0015] According to the benchmark parameters of the throughput indicator and the number of calls of each business component, the first evaluation algorithm is used to calculate a reference value of the throughput indicator of each business component.

[0016] Optionally, the benchmark parameters of the throughput indicator include: benchmark parameters for multiple transaction types, and the number of calls includes: the number of calls for the multiple transaction types;

[0017] The step of calculating a reference value of the throughput indicator of each service component by using the first evaluation algorithm according to the benchmark parameter of the throughput indicator and the number of calls of each service component includes:

[0018] Calculate the throughput index parameters of each business component for the multiple transaction types using the first evaluation algorithm according to the benchmark parameters of the multiple transaction types and the call times of the multiple transaction types;

[0019] According to the throughput indicator parameters of each business component for the multiple transaction types, the first evaluation algorithm is used to calculate a reference value of the throughput indicator of each business component.

[0020] Optionally, the benchmark parameters of the at least two performance indicators further include: a benchmark parameter of a response time indicator, the evaluation algorithm of the at least two performance indicators further includes: a second evaluation algorithm, the component call information of each business component includes: the number of calls and the response time ratio, the response time ratio is used to characterize the ratio of the response time of each business component to the total response time of the multiple business components;

[0021] The step of calculating the reference performance indicator parameters of each business component by using the evaluation algorithms of the at least two performance indicators respectively according to the benchmark parameters of the at least two performance indicators and the component call information of each business component includes:

[0022] According to the benchmark parameters of the response time indicator and the number of calls and response time proportion of each business component, the second evaluation algorithm is used to calculate the reference value of the response time indicator of each business component.

[0023] Optionally, obtaining component call information of multiple business components in the newly created business system includes:

[0024] The link call model of the target transaction type in the preset transaction test state is adopted to obtain the component call information of the multiple business components from the test execution result of the newly created business system.

[0025] Optionally, the test execution result of the newly created business system is a test execution result obtained by executing the following steps in the preset transaction test state:

[0026] Receiving a test transaction request of the target transaction type in the preset transaction test state;

[0027] According to the target transaction type and the transaction link corresponding to the target transaction type, the multiple business components are called to execute the test transaction request to obtain the test execution result of the newly created business system.

[0028] In a second aspect, an embodiment of the present application further provides a system performance evaluation device, the device comprising:

[0029] An acquisition module, used to acquire the benchmark performance indicator parameters of the newly created business system and component call information of multiple business components in the newly created business system, wherein the component call information is used to indicate the call status of the corresponding business component in a preset transaction test state;

[0030] A calculation module is used to calculate reference performance indicator parameters of the multiple business components based on the benchmark performance indicator parameters and the component call information, using a preset system evaluation model; the reference performance indicator parameters of the multiple business components are used to provide reference performance indicators for updating the multiple business components.

[0031] Optionally, the benchmark performance indicator parameters include: benchmark parameters of at least two performance indicators; the preset system evaluation model includes: evaluation algorithms of the at least two performance indicators; the reference performance indicator parameters include: reference values ​​for the at least two performance indicators;

[0032] The calculation module is specifically used to calculate the reference value of each business component for the at least two performance indicators by using the evaluation algorithms of the at least two performance indicators according to the benchmark parameters of the at least two performance indicators and the component call information of each business component.

[0033] Optionally, the baseline parameters of the at least two performance indicators include: a baseline parameter of the throughput indicator, the evaluation algorithms of the at least two performance indicators include: a first evaluation algorithm, and the component call information of each business component includes: the number of calls; the calculation module is specifically used to calculate the reference value of the throughput indicator of each business component according to the baseline parameters of the throughput indicator and the number of calls of each business component, using the first evaluation algorithm.

[0034] Optionally, the benchmark parameters of the throughput indicator include: benchmark parameters for multiple transaction types, and the number of calls includes: the number of calls for the multiple transaction types; the calculation module is specifically used to calculate the throughput indicator parameters of each business component for the multiple transaction types based on the benchmark parameters of the multiple transaction types and the number of calls for the multiple transaction types, using the first evaluation algorithm; and calculate the reference value of the throughput indicator of each business component based on the throughput indicator parameters of each business component for the multiple transaction types, using the first evaluation algorithm.

[0035] Optionally, the benchmark parameters of the at least two performance indicators also include: a benchmark parameter of a response time indicator, and the evaluation algorithm of the at least two performance indicators also includes: a second evaluation algorithm, the component call information of each business component includes: the number of calls and the response time ratio, and the response time ratio is used to characterize the ratio of the response time of each business component to the total response time of the multiple business components; the calculation module is also used to calculate the reference value of the response time indicator of each business component according to the benchmark parameters of the response time indicator and the number of calls and the response time ratio of each business component, using the second evaluation algorithm.

[0036] Optionally, the acquisition module is further configured to adopt a link call model of a target transaction type in the preset transaction test state to acquire component call information of the multiple business components from a test execution result of the newly created business system.

[0037] Optionally, the device further comprises:

[0038] A receiving module, configured to receive a test transaction request of the target transaction type under the preset transaction test state;

[0039] The execution module is used to call the multiple business components to execute the test transaction request according to the target transaction type and the transaction link corresponding to the target transaction type, and obtain the test execution result of the new business system.

[0040] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the system performance evaluation method as described in any one of the first aspects.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the system performance evaluation method as described in any one of the first aspects are executed.

[0042] The beneficial effects of this application are:

[0043] The system performance evaluation method, device, electronic device and storage medium provided in the present application determine reference performance indicator parameters of multiple business components based on the baseline performance indicator parameters of the newly built business system and the component call information of multiple business components. Compared with evaluating the performance of each business component of the newly built business system with an existing production system or based on experience, this solution can determine accurate reference performance indicator parameters for each business component, thereby accurately evaluating the performance indicators of each business component, so as to update or optimize each business component and the entire newly built business system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 Schematic diagram of the process of the system performance evaluation method provided in the embodiment of the present application Figure 1 ;

[0046] Figure 2 Schematic diagram of the process of the system performance evaluation method provided in the embodiment of the present application Figure 2 ;

[0047] Figure 3 Schematic diagram of the process of the system performance evaluation method provided in the embodiment of the present application Figure 3 ;

[0048] Figure 4 A schematic diagram of the structure of a system performance evaluation device provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0051] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0052] In addition, the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0054] In the prior art, when a bank has a new business system that needs to be launched, it can only refer to the existing production system or evaluate the reference performance indicators that should be met for each business component of the new business system based on personal experience. However, the reference performance indicators determined in this way are not accurate. To this end, the present application provides a system performance evaluation method, device, electronic device and storage medium, which determine the reference performance indicator parameters of multiple business components based on the baseline performance indicator parameters of the new business system and the component call information of multiple business components, so as to evaluate each business component and the entire new business system based on the reference performance indicator parameters, so that each business component and the entire new business system can be updated or optimized.

[0055] Figure 1 Schematic diagram of the process of the system performance evaluation method provided in the embodiment of the present application Figure 1 ,like Figure 1 As shown, the method may include:

[0056] S101: Obtaining benchmark performance indicator parameters of a newly created business system and component call information of multiple business components in the newly created business system, where the component call information is used to indicate the call status of the corresponding business component under a preset transaction test state.

[0057] In this embodiment, the newly created business system is a business system that is not online. The newly created business system is composed of multiple business components or multiple business subsystems. After receiving a transaction request, the newly created business system calls one or more business components related to the transaction request to execute the transaction request and obtain a transaction result.

[0058] For example, the multiple business components included in the newly created business system are: online gateway component, transaction middle platform, online engine component, distributed transaction component, hot resource component, sequence component, message component, parameter cache component, external gateway component, business database component, etc.

[0059] The benchmark performance indicator parameters are the standards that the entire new business system needs to meet in advance when designing a new business system. The component call information is the call information and operation information of each business component when the new business system calls each business component to execute a test transaction request under a preset transaction test state.

[0060] Among them, the preset transaction test state refers to the process of testing the functions of the newly built business system before the new business system is launched. In the preset transaction test state, a test transaction request is created and processed by the newly built business system to simulate the process of the new business system processing real transaction requests after it is launched.

[0061] It should be noted that, in the preset transaction test state, the hardware resources of the hardware equipment where the new business system is located need to be consistent with the hardware resources of the hardware equipment deployed after the new business system goes online.

[0062] S102: Calculate reference performance indicator parameters of multiple business components using a preset system evaluation model based on the benchmark performance indicator parameters and component call information; the reference performance indicator parameters of the multiple business components are used to provide reference performance indicators for updating the multiple business components.

[0063] In this embodiment, the preset system evaluation model is a pre-created mathematical model that can calculate the reference performance indicator parameters of business components. According to the component calling information of each business component, the calling information and operation information of the business component in the process of executing a test transaction request can be determined. The benchmark performance indicator parameters and the component calling information of each component are input into the preset system evaluation model, and the reference performance indicator parameters of each component are output respectively.

[0064] In some embodiments, under a preset transaction test state, multiple reference performance indicator parameters are required to be obtained through a large number of testing processes to determine the reference performance indicator parameters of each business component. Specifically, the reference performance indicator parameters of each business component can be determined based on the average value, maximum value, etc. of multiple reference performance indicator parameters calculated based on multiple test transaction requests.

[0065] The reference performance indicator parameters are the performance indicator parameters that the business components need to meet. Designers can evaluate the performance indicators of the business components based on the actual performance indicator parameters of the business components and the reference performance indicator parameters to determine whether the business components meet the requirements of the reference performance indicator parameters, thereby determining whether the entire new business system can meet the requirements of the benchmark performance indicator parameters. Furthermore, the business components can be updated or optimized based on the reference performance indicator parameters, thereby updating or optimizing the entire new business system.

[0066] The system performance evaluation method provided in the above embodiment determines the reference performance indicator parameters of multiple business components based on the baseline performance indicator parameters of the newly built business system and the component call information of multiple business components. Compared with evaluating the performance of each business component of the newly built business system based on an existing production system or based on experience, this solution can determine the accurate reference performance indicator parameters of each business component, thereby accurately evaluating the performance indicators of each business component, so as to update or optimize each business component and the entire newly built business system.

[0067] In a possible implementation, the benchmark performance indicator parameters include: benchmark parameters of at least two performance indicators; the preset system evaluation model includes: evaluation algorithms of at least two performance indicators; the reference performance indicator parameters include: reference values ​​for at least two performance indicators; the above S102, based on the benchmark performance indicator parameters and the component call information, using the preset system evaluation model to calculate the reference performance indicator parameters of multiple business components, may include:

[0068] According to the benchmark parameters of at least two performance indicators and the component calling information of each business component, evaluation algorithms of at least two performance indicators are respectively adopted to calculate the reference value of each business component for the at least two performance indicators.

[0069] In this embodiment, the performance of a newly built business system needs to be evaluated from multiple dimensions. Therefore, the benchmark performance indicator parameters may include benchmark parameters of at least two performance indicators. Correspondingly, the preset system evaluation model may also include an evaluation algorithm related to at least two performance indicators. According to the benchmark parameters of each performance indicator and the calling information of each business component, an evaluation algorithm corresponding to each performance indicator is used to calculate the reference value of each business component for each performance indicator. The reference performance indicator parameters of each business component may include reference values ​​of at least two performance indicators.

[0070] In some embodiments, the component calling information of the business components involved in the evaluation algorithm of each performance indicator may be different component calling information.

[0071] The system performance evaluation method provided in the above embodiment uses at least two performance indicator evaluation algorithms respectively through the baseline parameters of at least two performance indicators and the component call information of each business component to calculate the reference value of each business component for at least two performance indicators, thereby determining the reference values ​​of the performance indicators of multiple dimensions that the business components need to meet, so as to more accurately evaluate the performance indicators of each business component in multiple dimensions and improve the effect of optimizing business components and creating new business systems.

[0072] In a possible implementation, the benchmark parameters of the at least two performance indicators include: a benchmark parameter of a throughput indicator, the evaluation algorithms of the at least two performance indicators include: a first evaluation algorithm, and the component call information of each business component includes: the number of calls; the process of calculating the reference value of each business component for the at least two performance indicators by using the evaluation algorithms of the at least two performance indicators respectively according to the benchmark parameters of the at least two performance indicators and the component call information of each business component may include:

[0073] According to the benchmark parameters of the throughput indicator and the number of calls of each service component, a first evaluation algorithm is used to calculate a reference value of the throughput indicator of each service component.

[0074] In this embodiment, the benchmark parameters of the performance indicators may include the benchmark parameters of the throughput indicators, that is, the average daily transaction volume of the newly built business system, which is used to evaluate the maximum number of transactions per second (TPS) that the newly built business system needs to meet. The component call information of each business component may be the number of calls, and the number of calls is the number of times the business component is called when the newly built business system executes a test transaction request. The evaluation algorithm of the throughput indicator is the first evaluation algorithm.

[0075] According to the benchmark parameters of the throughput indicator and the number of calls of each business component, the first evaluation algorithm is used to calculate the reference value of the throughput indicator of each business component. The reference value of the throughput indicator of each business component is used to indicate the throughput or TPS that each business component needs to meet in the process of processing transaction requests by the newly created business system.

[0076] In one possible implementation, Figure 2 Schematic diagram of the process of the system performance evaluation method provided in the embodiment of the present application Figure 2 ,like Figure 2 As shown, the process of calculating the reference value of the throughput indicator of each service component by using the first evaluation algorithm according to the benchmark parameters of the throughput indicator and the number of calls of each service component may include:

[0077] S201: Calculate throughput index parameters of each business component for the multiple transaction types using a first evaluation algorithm according to the benchmark parameters of the multiple transaction types and the call times of the multiple transaction types.

[0078] S202: Calculate a reference value of the throughput indicator of each business component using a first evaluation algorithm according to the throughput indicator parameters of each business component for multiple transaction types.

[0079] In this embodiment, the newly created business system can provide the function of processing transaction requests for multiple transaction types. During the actual online operation of the newly created business system, each business component in the newly created business system may need to process transaction requests of multiple transaction types at the same time. To this end, it is necessary to determine the throughput requirements that the business components must meet to simultaneously process transaction requests of multiple transaction types.

[0080] Specifically, based on the transaction volume proportions of multiple transaction types and benchmark parameters of throughput indicators, benchmark parameters for multiple transaction types are determined, and based on the benchmark parameters of each transaction type and the number of calls of each transaction type, the first evaluation algorithm is used to calculate the throughput indicator parameters of each business component for each transaction type. Then, the first evaluation algorithm is used to calculate the sum of the throughput indicator parameters of each business component for multiple transaction types to determine the reference value of the throughput indicator of each business component.

[0081] For example, the calculation formula of the throughput index parameter of each business component for each transaction type can be expressed as: throughput index parameter = benchmark parameter of each transaction type * number of calls of each transaction type.

[0082] In some embodiments, the transaction volume proportions of multiple transaction types may be set by a designer, or the transaction volume proportions of multiple transaction types may be determined based on multiple types of transaction volumes processed by an existing production system.

[0083] Furthermore, the newly created business system includes multiple transaction types, but the vast majority of transaction requests are distributed in some transaction types, and the remaining transaction types only involve very few transaction requests. In this case, the throughput requirements of the business components for the transaction types with a small number of transaction requests can be ignored.

[0084] Therefore, TopN transaction types can be selected from multiple transaction types, and benchmark parameters for TopN transaction types can be determined based on the transaction volume share of TopN transaction types and benchmark parameters of throughput indicators. The first evaluation algorithm can be used to calculate the throughput indicator parameters of each business component for each transaction type based on the benchmark parameters of each transaction type and the number of calls of each transaction type. Then, the first evaluation algorithm can be used to calculate the sum of the throughput indicator parameters of each business component for TopN transaction types to determine the reference value of the throughput indicator of each business component.

[0085] For example, Table 1 is a reference value of the throughput index of multiple business components provided in an embodiment of the present application. As shown in Table 1, the transaction types are financial transaction types and non-financial transaction types. The financial transaction types mainly involve transfer-in transaction types and transfer-out transaction types. The non-financial transaction types mainly involve account query types. The transaction volume of transfer-in transaction types and transfer-out transaction types accounts for 10% respectively, and the transaction volume of account query types accounts for 80%. The benchmark parameter TPS of the throughput index is 100000, then the benchmark parameters of the transfer-in transaction type and the transfer-out transaction type are 10000 respectively, and the benchmark parameter of the account query type is 80000. For example, if the number of calls to the online engine component of the transfer-in transaction type is 1, the throughput index parameter of the online engine component for the transfer-in transaction type is 10000*1=10000, the number of calls to the online engine component of the transfer-out transaction type is 1, the throughput index parameter of the online engine component for the transfer-out transaction type is 10000*1=10000, the number of calls to the online engine component of the account query type is 1, the throughput index parameter of the online engine component for the account query type is 80000*1=80000, then the reference value of the throughput index of the online engine component = 10000+10000+80000=100000.

[0086] Table 1 Reference values ​​of throughput indicators of multiple business components

[0087]

[0088]

[0089] In a possible implementation, the benchmark parameters of at least two performance indicators further include: a benchmark parameter of a response time indicator, the evaluation algorithm of at least two performance indicators further includes: a second evaluation algorithm, the component call information of each business component includes: the number of calls and the response time ratio, and the response time ratio is used to characterize the ratio of the response time of each business component to the total response time of multiple business components; the above-mentioned process of calculating the reference value of each business component for at least two performance indicators by using the evaluation algorithm of at least two performance indicators respectively according to the benchmark parameters of at least two performance indicators and the component call information of each business component may include:

[0090] According to the benchmark parameters of the response time indicator and the number of calls and response time proportion of each business component, a second evaluation algorithm is used to calculate the reference value of the response time indicator of each business component.

[0091] In this embodiment, the benchmark parameters of the performance indicators may also include benchmark parameters of the response time indicators, that is, the transaction time consumed by the newly built business system to process transaction requests, which is used to evaluate the transaction speed that the newly built business system needs to meet. The component call information of each business component may include the number of calls and the response time ratio. The response time ratio is the ratio of the response time of a business component to the total response time during the processing of a test transaction request. The evaluation algorithm of the response time indicator is the second evaluation algorithm.

[0092] According to the benchmark parameters of the response time indicator, the number of calls of each business component and the response time ratio, the second evaluation algorithm is used to calculate the reference value of the response time indicator of each business component.

[0093] The reference value of the response time indicator of each business component may be a reference value of a separate response time indicator of each business component for each transaction type, or may be the sum of the reference values ​​of the response time indicators of each business component for multiple transaction types.

[0094] In some embodiments, the calculation formula of the reference value of the response time indicator of each business component can be expressed as: the reference value of the response time indicator of each business component = (benchmark parameter of the response time indicator*response time ratio) / number of calls.

[0095] Furthermore, due to the difference between the test network environment of the newly built business system and the actual production network environment after the newly built business system goes online, the network delay of the two affects the reference value of the response time indicator of the business component. Therefore, the response time can be corrected according to the network delay difference between the two when calculating the response time ratio to ensure that the response time ratio is closer to the actual production network environment.

[0096] For example, if the network delay of the test network environment is greater than that of the production network environment, the network delay difference is added to the response time and total response time of the business component. If the network delay of the test network environment is less than that of the production network environment, the network delay difference is subtracted from the response time and total response time of the business component.

[0097] In some embodiments, the calculation formula for the reference value of the response time indicator of each business component can be expressed as: the reference value of the response time indicator of each business component = ((benchmark parameter of the response time indicator + F1-F2)*response time ratio) / number of calls, wherein F1 = test environment network delay-production environment network delay, F2 = production environment network delay*number of service calls, the number of service calls is the number of times the business component is called across transaction types, and the cross-transaction type is to process a test transaction request of the first transaction type, which requires calling the transaction processing flow of the second transaction type. For example, a transaction query may be required in a transfer business.

[0098] By deducting the response time delay caused by the number of cross-service calls from the reference value of the response time indicator of each business component, the response time of the business component itself can be determined to improve accuracy.

[0099] In other embodiments, the reference value of the response time indicator of each service component is modified directly according to the network delay difference.

[0100] It should be noted that the calculation of the reference value for the throughput indicator and the calculation of the reference value for the response time indicator cannot be based on one test transaction request, but should be determined based on multiple test transaction requests.

[0101] For example, Table 2 is a reference value of the response time indicator of each business component provided in an embodiment of the present application.

[0102] Table 2 Reference values ​​of response time indicators of various business components

[0103]

[0104]

[0105] In a possible implementation, the process of obtaining component call information of multiple business components in the newly created business system in S101 may include:

[0106] A link call model of a target transaction type under a preset transaction test state is used to obtain component call information of multiple business components from the test execution results of the newly created business system.

[0107] In this embodiment, multiple transaction types are tested separately under a preset transaction test state. The link call model is used to indicate the type of business components and the call process that the newly created business system needs to call when processing transaction requests. Each transaction type has a unique corresponding link call model. When the newly created business system executes a transaction request of each transaction type, a test execution result is generated. The test execution result is a test log. The link call model is used to parse the test execution result, and the call times of multiple business components, as well as the response time and total response time of each business component are determined from the test execution result.

[0108] In one possible implementation, Figure 3 Schematic diagram of the process of the system performance evaluation method provided in the embodiment of the present application Figure 3 ,like Figure 3 As shown, the test execution result of the newly created business system is the test execution result obtained by executing the following steps in the preset transaction test state:

[0109] S301: Receive a test transaction request of a target transaction type in a preset transaction test state.

[0110] S302: According to the target transaction type and the transaction link corresponding to the target transaction type, multiple business components are called to execute the test transaction request to obtain the test execution result of the newly created business system.

[0111] In this embodiment, under the preset transaction test state, the gateway of the newly created business system receives a test transaction request of the target transaction type and forwards the test transaction request to the transaction middle station. The transaction middle station determines the transaction link corresponding to the target transaction type based on the target transaction type of the test transaction request. The transaction link indicates the type of business components that need to be called when processing the test transaction request and the calling process. The transaction middle station calls the corresponding business components according to the transaction link to generate a test execution result based on the execution result of the business component.

[0112] In some embodiments, the transaction middle platform converts the test transaction request to the transaction component according to the target transaction type, and the transaction component calls the corresponding business component according to the transaction link. The transaction component is divided into a corporate transaction middle platform and a retail transaction middle platform. The corporate transaction middle platform is used to process corporate transaction requests, and the retail transaction middle platform is used to process retail transaction requests.

[0113] Based on the above method embodiment, the embodiment of the present application also provides a system performance evaluation device. Figure 4 A schematic diagram of the structure of the system performance evaluation device provided in the embodiment of the present application, such as Figure 4 As shown, the device may include:

[0114] The acquisition module 401 is used to acquire the benchmark performance index parameters of the newly created business system and the component call information of multiple business components in the newly created business system, where the component call information is used to indicate the call status of the corresponding business component in the preset transaction test state;

[0115] The calculation module 402 is used to calculate the reference performance indicator parameters of multiple business components according to the benchmark performance indicator parameters and component call information, using a preset system evaluation model; the reference performance indicator parameters of multiple business components are used to provide reference performance indicators for the update of multiple business components.

[0116] Optionally, the benchmark performance indicator parameters include: benchmark parameters of at least two performance indicators; the preset system evaluation model includes: evaluation algorithms of at least two performance indicators; the reference performance indicator parameters include: reference values ​​for at least two performance indicators;

[0117] The calculation module 402 is specifically used to calculate the reference value of each business component for at least two performance indicators by using evaluation algorithms of at least two performance indicators according to the benchmark parameters of at least two performance indicators and the component call information of each business component.

[0118] Optionally, the baseline parameters of at least two performance indicators include: the baseline parameters of the throughput indicator, the evaluation algorithms of at least two performance indicators include: the first evaluation algorithm, and the component call information of each business component includes: the number of calls; the calculation module 402 is specifically used to calculate the reference value of the throughput indicator of each business component according to the baseline parameters of the throughput indicator and the number of calls of each business component, using the first evaluation algorithm.

[0119] Optionally, the benchmark parameters of the throughput indicator include: benchmark parameters for multiple transaction types, and the number of calls includes: the number of calls for multiple transaction types; the calculation module 402 is specifically used to calculate the throughput indicator parameters of each business component for multiple transaction types based on the benchmark parameters of the multiple transaction types and the number of calls for the multiple transaction types, using a first evaluation algorithm; and calculate the reference value of the throughput indicator of each business component based on the throughput indicator parameters of each business component for multiple transaction types, using the first evaluation algorithm.

[0120] Optionally, the baseline parameters of at least two performance indicators also include: a baseline parameter of a response time indicator, and the evaluation algorithm of at least two performance indicators also includes: a second evaluation algorithm, the component call information of each business component includes: the number of calls and the response time ratio, and the response time ratio is used to characterize the ratio of the response time of each business component to the total response time of multiple business components; the calculation module 402 is also used to calculate the reference value of the response time indicator of each business component according to the baseline parameters of the response time indicator and the number of calls and the response time ratio of each business component, using the second evaluation algorithm.

[0121] Optionally, the acquisition module 401 is further configured to adopt a link call model of a target transaction type in a preset transaction test state to acquire component call information of multiple business components from the test execution results of the newly created business system.

[0122] Optionally, the device may further include:

[0123] A receiving module, used for receiving a test transaction request of a target transaction type under a preset transaction test state;

[0124] The execution module is used to call multiple business components to execute the test transaction request according to the target transaction type and the transaction link corresponding to the target transaction type, and obtain the test execution result of the newly created business system.

[0125] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0126] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0127] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application, such as Figure 5As shown, the electronic device 500 includes: a processor 501, a storage medium 502 and a bus, wherein the storage medium 502 stores program instructions executable by the processor 501, and when the electronic device 500 is running, the processor 501 communicates with the storage medium 502 via the bus, and the processor 501 executes the program instructions to execute the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0128] Optionally, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program executes the above method embodiment when executed by a processor.

[0129] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0130] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0131] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0132] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0133] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A system performance evaluation method, characterized in that: The method comprises: Obtaining a benchmark performance indicator parameter of a newly created business system and component call information of multiple business components in the newly created business system, wherein the component call information is used to indicate a call status of a corresponding business component in a preset transaction test state; According to the benchmark performance indicator parameters and the component call information, a preset system evaluation model is used to calculate reference performance indicator parameters of the multiple business components; the reference performance indicator parameters of the multiple business components are used to provide reference performance indicators for updating the multiple business components.

2. The method according to claim 1, characterized in that The benchmark performance indicator parameters include: benchmark parameters of at least two performance indicators; the preset system evaluation model includes: evaluation algorithms of the at least two performance indicators; the reference performance indicator parameters include: reference values ​​for the at least two performance indicators; The step of calculating reference performance indicator parameters of the plurality of business components using a preset system evaluation model according to the benchmark performance indicator parameters and the component call information includes: According to the benchmark parameters of the at least two performance indicators and the component calling information of each business component, the evaluation algorithms of the at least two performance indicators are respectively adopted to calculate the reference value of each business component for the at least two performance indicators.

3. The method according to claim 2, characterized in that The benchmark parameters of the at least two performance indicators include: a benchmark parameter of a throughput indicator, the evaluation algorithms of the at least two performance indicators include: a first evaluation algorithm, and the component call information of each business component includes: the number of calls; The step of calculating the reference performance indicator parameters of each business component by respectively using the evaluation algorithms of the at least two performance indicators according to the benchmark parameters of the at least two performance indicators and the component call information of each business component includes: According to the benchmark parameters of the throughput indicator and the number of calls of each business component, the first evaluation algorithm is used to calculate a reference value of the throughput indicator of each business component.

4. The method according to claim 3, characterized in that The benchmark parameters of the throughput indicator include: benchmark parameters for multiple transaction types, and the number of calls includes: the number of calls for the multiple transaction types; The step of calculating the reference value of the throughput indicator of each service component by using the first evaluation algorithm according to the benchmark parameter of the throughput indicator and the number of calls of each service component includes: Calculate the throughput index parameters of each business component for the multiple transaction types using the first evaluation algorithm according to the benchmark parameters of the multiple transaction types and the call times of the multiple transaction types; According to the throughput indicator parameters of each business component for the multiple transaction types, the first evaluation algorithm is used to calculate a reference value of the throughput indicator of each business component.

5. The method according to claim 2, characterized in that The benchmark parameters of the at least two performance indicators further include: a benchmark parameter of a response time indicator, the evaluation algorithm of the at least two performance indicators further includes: a second evaluation algorithm, the component call information of each business component includes: the number of calls and the response time ratio, the response time ratio is used to characterize the ratio of the response time of each business component to the total response time of the multiple business components; The step of calculating the reference performance indicator parameters of each business component by respectively using the evaluation algorithms of the at least two performance indicators according to the benchmark parameters of the at least two performance indicators and the component call information of each business component includes: According to the benchmark parameters of the response time indicator and the number of calls and the response time proportion of each business component, the second evaluation algorithm is used to calculate the reference value of the response time indicator of each business component.

6. The method according to claim 1, characterized in that Obtaining component call information of multiple business components in the newly created business system, including: The link call model of the target transaction type in the preset transaction test state is adopted to obtain the component call information of the multiple business components from the test execution result of the newly created business system.

7. The method according to claim 6, characterized in that The test execution result of the newly created business system is the test execution result obtained by executing the following steps under the preset transaction test state: Receiving a test transaction request of the target transaction type in the preset transaction test state; According to the target transaction type and the transaction link corresponding to the target transaction type, the multiple business components are called to execute the test transaction request to obtain the test execution result of the newly created business system.

8. A system performance evaluation device, characterized in that: The device comprises: An acquisition module, used to acquire the benchmark performance indicator parameters of the newly created business system and component call information of multiple business components in the newly created business system, wherein the component call information is used to indicate the call status of the corresponding business component under a preset transaction test state; A calculation module is used to calculate reference performance indicator parameters of the multiple business components based on the benchmark performance indicator parameters and the component call information, using a preset system evaluation model; the reference performance indicator parameters of the multiple business components are used to provide reference performance indicators for updating the multiple business components.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the system performance evaluation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the system performance evaluation method according to any one of claims 1 to 7 are executed.

Images