A method and device for testing and evaluating service performance stability

Abstract

The present invention provides a method and device for evaluating a service performance stability test, the method comprising obtaining actual performance data after a service performance test lasts for a preset time and predicted performance data predicted by a prediction model, inputting the predicted performance data and the actual performance data into a loss function of the prediction model at preset time intervals, determining the accuracy of the prediction model, and if the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than a first time, using the prediction model to predict the predicted performance data within a preset time period of the service performance test, and evaluating the service performance test. By using highly accurate predicted performance data for service performance stability testing, the test time of the stability test can be greatly shortened and the efficiency of the stability test can be improved.

Description

Technical Field

[0001] The present invention relates to the field of financial technology (Fintech), and in particular to a method and device for testing and evaluating service performance stability. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. Traditional finance is gradually transforming into FinTech. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology. During the system stability testing process in the financial field, service performance testing is a key issue.

[0003] Service stability testing (reliability testing) requires over 12 hours to generate test data. Testers determine test success based on service processing throughput, response time, CPU utilization, memory utilization, network I / O, and other metrics. Each stress test of the solution requires over 12 hours. With the faster pace of development, testing becomes a bottleneck in the implementation of requirements. Furthermore, many project management scenarios do not support extended testing, resulting in a lack of stability assessment for the release. Summary of the Invention

[0004] The present invention provides a method and device for testing and evaluating service performance stability, which can save time for stability testing and accelerate the system development process.

[0005] In a first aspect, the present invention provides a method for testing and evaluating service performance stability, comprising:

[0006] Acquire actual performance data of a service performance test after a preset period of time and predicted performance data of the service performance test predicted by a prediction model; the prediction model is trained and learned from historical performance data of a service performance stability test;

[0007] Inputting the predicted performance data and the actual performance data into a loss function of the prediction model at preset time intervals to determine the accuracy of the prediction model;

[0008] If the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than the first time, the prediction model is used to predict the predicted performance data within the preset time period of the service performance test; and the service performance test is evaluated.

[0009] In the above technical solution, by using highly accurate predicted performance data to perform service performance stability testing, the testing time of the stability test can be greatly shortened and the efficiency of the stability test can be improved.

[0010] Optionally, the step of training and learning historical performance data of the service performance stability test to obtain the prediction model includes:

[0011] Obtaining historical performance data of the service performance stability test;

[0012] Normalizing the historical performance data to obtain a normalized data sequence;

[0013] Dividing the normalized data sequence into a training set and a test set;

[0014] The preset gated recurrent unit model is trained and learned based on the training set and the test set to obtain the prediction model.

[0015] In the above technical solution, by training and learning historical performance data, a prediction model that can predict service performance test data can be obtained.

[0016] Optionally, the training and learning of a preset gated recurrent unit model based on the training set and the test set to obtain the prediction model includes:

[0017] Inputting the training set into the preset gated recurrent unit model for forward propagation to obtain a predicted value;

[0018] Evaluate the preset gated recurrent unit model using a mean square error loss function based on the predicted value and the test set, and determine an update gradient of the loss function with respect to the parameters;

[0019] Backpropagation is performed on the preset gated recurrent unit model according to the updated gradient of the loss function on the parameters; the next round of training is continued until the preset gated recurrent unit model converges to obtain the prediction model.

[0020] In the above technical solution, the preset gated recurrent unit model is trained and learned through the training set and the test set, which can speed up the training process of the prediction model and improve the training efficiency.

[0021] Optionally, inputting the training set into the preset gated recurrent unit model for forward propagation to obtain a predicted value includes:

[0022] Inputting each data sequence in the training set into the preset gated recurrent unit model, and resetting each data sequence according to a reset gate of the preset gated recurrent unit model to obtain reset data;

[0023] Use a hyperbolic tangent activation function to scale the reset data and its corresponding data sequence to obtain the current state;

[0024] Using the update gate of the preset gated recurrent unit model to update the state at the current moment to obtain a hidden state to be passed to the next node;

[0025] The predicted value is obtained according to the hidden state transferred to the next node.

[0026] In the above technical solution, due to the calculation of the reset gate and the update gate in the preset gated recurrent unit model, the time for determining the prediction value is saved, and the efficiency of model training is further accelerated.

[0027] Optionally, the evaluating the service performance test includes:

[0028] Perform trend analysis on the predicted performance data within the preset time period to obtain the trend corresponding to the predicted performance data within the preset time period; input the trend corresponding to the predicted performance data within the preset time period into the stability risk assessment rule engine to obtain the risk of the service performance stability test assessment.

[0029] In the above technical solution, by inputting the trend corresponding to the predicted performance data into the stability risk assessment rule engine, the risk of the service performance stability test can be automatically assessed.

[0030] Optionally, the method further includes:

[0031] If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is less than the first time, evaluating the prediction model at the next preset time interval;

[0032] If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is greater than the second time, it is determined that the prediction model cannot be evaluated.

[0033] Optionally, the performance data includes service performance data and hardware performance data;

[0034] The service performance data includes TPS, response time, error rate, concurrency, and number of instances;

[0035] The emergency performance data includes CPU usage, memory usage, hard disk read and write rate, and network speed.

[0036] In a second aspect, an embodiment of the present invention provides a device for testing and evaluating service performance stability, including:

[0037] an acquisition unit, configured to acquire actual performance data of a service performance test after a preset period of time and predicted performance data of the service performance test predicted by a prediction model; the prediction model is obtained by training and learning historical performance data of a service performance stability test;

[0038] A processing unit is used to input the predicted performance data and the actual performance data into the loss function of the prediction model at preset time intervals to determine the accuracy of the prediction model; if the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than the first time, use the prediction model to predict the predicted performance data within the preset time period of the service performance test; and evaluate the service performance test.

[0039] Optionally, the processing unit is specifically configured to:

[0040] Obtaining historical performance data of the service performance stability test;

[0041] Normalizing the historical performance data to obtain a normalized data sequence;

[0042] Dividing the normalized data sequence into a training set and a test set;

[0043] The preset gated recurrent unit model is trained and learned based on the training set and the test set to obtain the prediction model.

[0044] Optionally, the processing unit is specifically configured to:

[0045] Inputting the training set into the preset gated recurrent unit model for forward propagation to obtain a predicted value;

[0046] Evaluate the preset gated recurrent unit model using a mean square error loss function based on the predicted value and the test set, and determine an update gradient of the loss function with respect to the parameters;

[0047] Backpropagation is performed on the preset gated recurrent unit model according to the updated gradient of the loss function on the parameters; the next round of training is continued until the preset gated recurrent unit model converges to obtain the prediction model.

[0048] Optionally, the processing unit is specifically configured to:

[0049] Inputting each data sequence in the training set into the preset gated recurrent unit model, and resetting each data sequence according to a reset gate of the preset gated recurrent unit model to obtain reset data;

[0050] Use a hyperbolic tangent activation function to scale the reset data and its corresponding data sequence to obtain the current state;

[0051] Using the update gate of the preset gated recurrent unit model to update the state at the current moment to obtain a hidden state to be passed to the next node;

[0052] The predicted value is obtained according to the hidden state transferred to the next node.

[0053] Optionally, the processing unit is specifically configured to:

[0054] Perform trend analysis on the predicted performance data within the preset time period to obtain the trend corresponding to the predicted performance data within the preset time period; input the trend corresponding to the predicted performance data within the preset time period into the stability risk assessment rule engine to obtain the risk of the service performance stability test assessment.

[0055] Optionally, the processing unit is further configured to:

[0056] If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is less than the first time, evaluating the prediction model at the next preset time interval;

[0057] If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is greater than the second time, it is determined that the prediction model cannot be evaluated.

[0058] Optionally, the performance data includes service performance data and hardware performance data;

[0059] The service performance data includes TPS, response time, error rate, concurrency, and number of instances;

[0060] The emergency performance data includes CPU usage, memory usage, hard disk read and write rate, and network speed.

[0061] In a third aspect, the present invention provides a computing device, comprising:

[0062] memory for storing computer programs;

[0063] The processor is configured to call the computer program stored in the memory and execute the method described in the first aspect according to the obtained program.

[0064] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] Figure 1 A schematic diagram of a system architecture provided by an embodiment of the present invention;

[0067] Figure 2 A flowchart of a method for testing and evaluating service performance stability provided by an embodiment of the present invention;

[0068] Figure 3 A flowchart of a method for testing and evaluating service performance stability provided by an embodiment of the present invention;

[0069] Figure 4 A schematic structural diagram of a device for testing and evaluating service performance stability provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some of the embodiments of the present invention, rather than all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0071] Figure 1 A system architecture is provided for an embodiment of the present invention. Figure 1 As shown, the system architecture may be a server 100 , including a processor 110 , a communication interface 120 and a memory 130 .

[0072] The communication interface 120 is used to communicate with the Kubernetes (open source container orchestration engine) platform, and to send and receive information transmitted by the Kubernetes platform to achieve communication.

[0073] The processor 110 is the control center of the server 100. It connects various parts of the server 100 using various interfaces and lines. It executes various functions of the server 100 and processes data by running or executing software programs and / or modules stored in the memory 130 and calling data stored in the memory 130. Optionally, the processor 110 may include one or more processing units.

[0074] Memory 130 can be used to store software programs and modules. Processor 110 executes various functional applications and data processing by running the software programs and modules stored in memory 130. Memory 130 may primarily include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the data storage area may store data generated based on business processing. Memory 130 may also include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state memory device.

[0075] It should be noted that the above Figure 1 The structure shown is only an example and is not limited in the embodiment of the present invention.

[0076] Based on the above description, Figure 2 The flow of a method for testing and evaluating service performance stability provided by an embodiment of the present invention is exemplarily shown. The flow can be executed by a device for testing and evaluating service performance stability.

[0077] like Figure 2 As shown, the specific steps of the process include:

[0078] Step 201 : Acquire actual performance data of a service performance test after a preset time period and predicted performance data of the service performance test predicted by a prediction model.

[0079] In the embodiment of the present invention, before starting the service performance test, it is necessary to first train the prediction model.

[0080] Specifically, the historical performance data of the service performance stability test can be obtained first, and then the historical performance data can be normalized to obtain a normalized data sequence. Finally, the normalized data sequence can be divided into a training set and a test set, and the preset gated recurrent unit (GRU) model can be trained and learned based on the training set and the test set to obtain the prediction model. Among them, when performing training and learning, the training set can be first input into the preset gated recurrent unit model for forward propagation to obtain the predicted value. The preset gated recurrent unit model is evaluated using the mean square error loss function based on the predicted value and the test set to determine the updated gradient of the loss function to the parameters. Finally, the preset gated recurrent unit model is back-propagated based on the updated gradient of the loss function to the parameters; until the preset gated recurrent unit model converges to obtain the prediction model. Among them, in the process of determining the predicted value, each data sequence in the training set can be first input into the preset gated recurrent unit model, and each data sequence can be reset according to the reset gate of the preset gated recurrent unit model to obtain the reset data, and then the reset data and its corresponding data sequence are scaled using the hyperbolic tangent activation function to obtain the state at the current moment, and finally the state at the current moment is updated using the update gate of the preset gated recurrent unit model to obtain the hidden state passed to the next node, and then the predicted value can be obtained according to the hidden state passed to the next node.

[0081] For example, we first use the Kubernetes platform Prometheus (an open-source monitoring system) to collect performance data related to historical service stability testing. Service performance data includes: TPS (Transaction Per Second), response time, error rate, concurrency, and number of instances; hardware performance data includes: CPU usage, memory usage, hard disk read / write rate, and network speed. The data is normalized and mapped to the range [0,1]. TPS, response time, concurrency, and other data are normalized using the formula x′ i =(x i -x min ) / (x max -x min ) processing, where x i is the data sequence to be normalized, x min is the maximum value in the data sequence to be processed, x max is the minimum value in the data sequence to be processed. The data sequence is divided into training set and test set.

[0082] In the implementation of the present invention, the GRU neural network model sets a reset gate r and an update gate z to forget and update the data generated by the previous node training step.

[0083] During training, the input of the reset gate and update gate is the input x of the current node t And the hidden state h passed down from the previous node t-1 ,h t-1 Contains the relevant information of the previous node. The forward propagation formula when using the training set for forward propagation is: t =σ(W z ·[h t-1 ,x t ]), r t =σ(W r ·[h t-1 ,x t ]). Where σ is the sigmoid activation function, W z 、W r is the weight parameter, h t-1 represents the state information at time t-1, x t Represents the input information at time t. Update gate z t The closer it is to 1, the more data is remembered, and the closer it is to 0, the more data is forgotten.

[0084] Get the reset data r through the reset gate t ·h t-1 , the data is scaled to (-1,1) through the tanh activation function, and h′ is obtained t =τanh(W·[r t ·h t-1 ,x t ]). h′ t Mainly includes x t Data is equivalent to selecting and memorizing the current state. The next step is to enter the update memory stage, using the update gate, h t =z·h t-1 +(1-z)·h′ t The h t It is represented as the hidden state passed to the next node. According to the hidden state passed to the next node, the output value of the model, that is, the predicted value y, can be obtained. t =σ(W0·h t ). In each formula, [] indicates that two vector matrices are connected, and · indicates the product of two vector matrices.

[0085] During the training process, each round of training requires the use of the mean squared error (MSE) loss function. Evaluate the degree of fit of the GRU model to the training data during the training phase. Where N represents the number of training data, y iRepresents the label of the i-th data, which is the true value, y′ i is the GRU prediction value. The smaller J(w) is, the higher the model evaluation prediction accuracy is.

[0086] Among them, you can choose ADAM (Adaptive Moment Estimation) training optimizer to calculate the gradient g of the mean square error loss function J(w) with respect to the parameter w t , First, the exponential moving average (first-order moment) of the gradient is calculated, and m0 is initialized to 0. t =β1·m t-1 +(1-β1)·g t , the β1 coefficient is the exponential decay rate, which controls the weight distribution of momentum and current gradient, and its value is 0.9. Then, the exponential moving average (second-order moment) of the square of the gradient is calculated, and v0 is initialized to 0. The β2 coefficient is an exponential decay rate that controls the influence of the previous gradient square, and its value is 0.999. Because m0=0, m t It is close to 0 in the early stage of training, so m t Perform bias correction, formula Similarly, v t The correction logic is

[0087] Based on the exponential moving average of the gradient and the exponential moving average of the gradient score, the back propagation of the J(w) model can be realized, that is, the update of the parameter w of the J(w) model can be achieved, where the update logic can be: The learning rate ɑ is set to 0.001, and ∈ is set to 1E-8 to avoid the denominator being 0. After the update of the J(w) model parameter w is completed, the next round of training can be continued until the model converges and a prediction model is obtained. In the embodiment of the present invention, by using the gradient determined by the ADAM optimizer to update the parameters in the J(w) model, the convergence speed of the model can be accelerated and the efficiency of model training can be improved.

[0088] By using the prediction model obtained through the above training process to make predictions, the predicted performance data of the service performance test can be obtained. That is, by inputting the prediction time into the prediction model, the predicted performance data corresponding to the prediction time can be obtained.

[0089] The actual performance data of the service performance test in the embodiment of the present invention refers to the performance data actually generated when the service performance is stress-tested, that is, the data after a preset time period, which can be set based on experience.

[0090] Step 202: Input the predicted performance data and the actual performance data into the loss function of the prediction model at preset time intervals to determine the accuracy of the prediction model.

[0091] After obtaining the predicted performance data and the actual performance data, the prediction model can be evaluated at preset time intervals to evaluate the accuracy of the prediction model.

[0092] The preset time interval can be set based on experience, for example, 5 minutes. That is, the accuracy is evaluated every 5 minutes, and the predicted performance data is compared with the actual performance data using a loss function to obtain the accuracy of the prediction model. Only when the accuracy of the prediction model is greater than the preset threshold can the prediction model be determined to be stable.

[0093] Step 203: If the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than the first time, the prediction model is used to predict the predicted performance data within the preset time period of the service performance test, and the service performance test is evaluated.

[0094] In an embodiment of the present invention, the preset threshold can be set based on experience, and the preset threshold is the fitness threshold used in the loss function evaluation during the above-mentioned prediction model training process. The first time can be set based on experience, for example, the first time can be set to 10 minutes. When the accuracy of the prediction model is higher than the preset threshold and the duration of the service performance test is greater than the first time, it indicates that the prediction model can achieve accurate predictions, that is, the prediction model can be used to predict the prediction data within the preset time period of the service performance test. The preset time period can be set based on experience, which is generally the time of the stability test. For example, it can be 12 hours, which is equivalent to directly predicting 12 hours of data for stability test evaluation. The first time is equivalent to the shortest evaluation time for evaluating the accuracy of the prediction model. Only when it is greater than the first time can it be indicated that the evaluation of the accuracy of the prediction model is valid.

[0095] If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is less than the first time, it indicates that the accuracy of the prediction model is not high and needs to be continuously evaluated. Therefore, the evaluation of the prediction model at the next preset time interval is continued, which is equivalent to evaluating the prediction model once every preset time interval until the accuracy is greater than the preset threshold or the duration is greater than the second time. In other words, when the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is greater than the second time, it can be determined that the prediction model cannot be evaluated and the second time needs to be increased. The second time can be set based on experience, and the second time is greater than the minimum evaluation time. The second time is equivalent to the longest evaluation time for evaluating the accuracy of the prediction model. If the accuracy of the prediction model is still lower than the preset threshold after exceeding the second time, it indicates that the accuracy evaluation of the prediction model has failed.

[0096] After obtaining the predicted performance data within the preset time period, trend analysis can be performed based on the predicted performance data within the preset time period to obtain the trend corresponding to the predicted performance data within the preset time period; the trend corresponding to the predicted performance data within the preset time period is input into the stability risk assessment rule engine to obtain the risk of the service performance stability test assessment.

[0097] That is to say, after obtaining the predicted performance data within the preset time period, trend analysis can be performed, that is, the predicted performance data within the preset time period is plotted into a trend curve in chronological order to obtain the trend corresponding to the predicted performance data, such as CPU trend, memory trend, disk IO trend, network IO trend, FPS trend, development time, test time, number of concurrencies and other input parameters. By inputting these input parameters into the risk assessment rule engine, multiple levels of risk such as high, medium and low can be output, that is, the risk of service performance stability test assessment can be obtained.

[0098] In order to better explain the embodiments of the present invention, the process of the service performance stability test and evaluation will be described below in a specific implementation scenario.

[0099] like Figure 3 The process shown in FIG. 1 includes:

[0100] Step 301: construct service performance serialization data (including CPU, memory, network IO, service throughput, response time, etc.) and normalize the data.

[0101] Step 302: Use the training set to train the initial model based on the GRU model by setting the loss function, reset gate, update gate, etc.

[0102] The model training process has been described in detail in the above embodiments and will not be repeated here.

[0103] Step 303: determine whether the initial model's fitness reaches a fitness threshold. If so, proceed to step 304; if not, proceed to step 302 to continue model training.

[0104] Step 304: Determine the formal prediction model.

[0105] When the degree of fit reaches the threshold, it can be determined that the initial model training is completed and the formal prediction model is obtained.

[0106] Step 305: Start the stability test, which lasts for 10 minutes.

[0107] Perform a stability test on service performance for 10 minutes.

[0108] Step 306 : Compare the predicted value with the actual value and evaluate the accuracy every 5 minutes.

[0109] After 10 consecutive times, the predicted value obtained by the prediction model can be compared with the actual value obtained from the Kubernetes platform. The prediction model evaluation is performed every 5 minutes to evaluate the accuracy of the prediction model.

[0110] Step 307 , determining whether the duration is greater than the preset first time and the accuracy is higher than the set threshold; if so, proceed to step 308 ; if not, proceed to step 311 .

[0111] Step 308: Output the predicted 12-hour performance data.

[0112] Use the forecast model to output performance data for the next 12 hours.

[0113] Step 309: Processing of stability test risk assessment rules.

[0114] The performance data is processed according to the stability test risk assessment rules to obtain the corresponding trend of the performance data.

[0115] Step 310, (high / medium / low) risk assessment test report.

[0116] By inputting the trend corresponding to the performance data into the risk assessment rule engine, a risk assessment test report can be obtained, in which the risk can include multiple levels such as high / medium / low.

[0117] Step 311 , determine whether the duration is greater than the second time, if so, proceed to step 312 , otherwise, proceed to step 306 .

[0118] Step 312: evaluation is not possible and a second time period needs to be added.

[0119] When the time is greater than the second time, and the accuracy is still lower than the preset threshold, it indicates that the evaluation cannot be completed within the second time, and the second time needs to be increased.

[0120] The embodiment of the present invention shows that the actual performance data after the service performance test lasts for a preset time and the predicted performance data predicted by the prediction model are obtained, the predicted performance data and the actual performance data are input into the loss function of the prediction model at preset time intervals, and the accuracy of the prediction model is determined. If the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than the first time, the prediction model is used to predict the predicted performance data within the preset time period of the service performance test, and the service performance test is evaluated. By using highly accurate predicted performance data for service performance stability testing, the test time of the stability test can be greatly shortened and the efficiency of the stability test can be improved.

[0121] Based on the same technical concept, Figure 4 The schematic diagram of the structure of a device for testing and evaluating service performance stability provided by an embodiment of the present invention is exemplarily shown. The device can execute the process of testing and evaluating service performance stability.

[0122] like Figure 4 As shown, the device specifically includes:

[0123] An acquisition unit 401 is configured to acquire actual performance data of a service performance test after a preset period of time and predicted performance data of the service performance test predicted by a prediction model; the prediction model is obtained by training and learning historical performance data of a service performance stability test;

[0124] The processing unit 402 is used to input the predicted performance data and the actual performance data into the loss function of the prediction model at preset time intervals to determine the accuracy of the prediction model; if the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than the first time, the prediction model is used to predict the predicted performance data within the preset time period of the service performance test; and the service performance test is evaluated.

[0125] Optionally, the processing unit 402 is specifically configured to:

[0126] Obtaining historical performance data of the service performance stability test;

[0127] Normalizing the historical performance data to obtain a normalized data sequence;

[0128] Dividing the normalized data sequence into a training set and a test set;

[0129] The preset gated recurrent unit model is trained and learned based on the training set and the test set to obtain the prediction model.

[0130] Optionally, the processing unit 402 is specifically configured to:

[0131] Inputting the training set into the preset gated recurrent unit model for forward propagation to obtain a predicted value;

[0132] Evaluate the preset gated recurrent unit model using a mean square error loss function based on the predicted value and the test set, and determine an update gradient of the loss function with respect to the parameters;

[0133] Backpropagation is performed on the preset gated recurrent unit model according to the updated gradient of the loss function on the parameters; the next round of training is continued until the preset gated recurrent unit model converges to obtain the prediction model.

[0134] Optionally, the processing unit 402 is specifically configured to:

[0135] Inputting each data sequence in the training set into the preset gated recurrent unit model, and resetting each data sequence according to a reset gate of the preset gated recurrent unit model to obtain reset data;

[0136] Use a hyperbolic tangent activation function to scale the reset data and its corresponding data sequence to obtain the current state;

[0137] Using the update gate of the preset gated recurrent unit model to update the state at the current moment to obtain a hidden state to be passed to the next node;

[0138] The predicted value is obtained according to the hidden state transferred to the next node.

[0139] Optionally, the processing unit 402 is specifically configured to:

[0140] Perform trend analysis on the predicted performance data within the preset time period to obtain the trend corresponding to the predicted performance data within the preset time period; input the trend corresponding to the predicted performance data within the preset time period into the stability risk assessment rule engine to obtain the risk of the service performance stability test assessment.

[0141] Optionally, the processing unit 402 is further configured to:

[0142] If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is less than the first time, evaluating the prediction model at the next preset time interval;

[0143] If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is greater than the second time, it is determined that the prediction model cannot be evaluated.

[0144] Optionally, the performance data includes service performance data and hardware performance data;

[0145] The service performance data includes TPS, response time, error rate, concurrency, and number of instances;

[0146] The emergency performance data includes CPU usage, memory usage, hard disk read and write rate, and network speed.

[0147] Based on the same technical concept, an embodiment of the present invention provides a computing device, including:

[0148] memory for storing computer programs;

[0149] The processor is used to call the computer program stored in the memory and execute the above-mentioned service performance stability test and evaluation method according to the obtained program.

[0150] Based on the same technical concept, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer-executable program. The computer-executable program is used to enable a computer to execute the above-mentioned service performance stability test and evaluation method.

[0151] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0155] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0156] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for testing and evaluating service performance stability, characterized in that: include: Acquire actual performance data of the service performance test after a preset time has passed and predicted performance data of the service performance test predicted by the prediction model; The prediction model is obtained by training and learning the historical performance data of the service performance stability test; Inputting the predicted performance data and the actual performance data into a loss function of the prediction model at preset time intervals to determine the accuracy of the prediction model; If the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than the first time, the prediction model is used to predict the predicted performance data within a preset time period of the service performance test; the preset time interval is less than the first time, and the first time is less than the preset time period; Performing trend analysis based on the predicted performance data within the preset time period to obtain a trend corresponding to the predicted performance data within the preset time period; The trend corresponding to the predicted performance data within the preset time period is input into the stability risk assessment rule engine to obtain the risk of the service performance stability test assessment.

2. The method according to claim 1, characterized in that The training and learning of the historical performance data of the service performance stability test to obtain the prediction model includes: Obtaining historical performance data of the service performance stability test; Normalizing the historical performance data to obtain a normalized data sequence; Dividing the normalized data sequence into a training set and a test set; The preset gated recurrent unit model is trained and learned according to the training set and the test set to obtain the prediction model.

3. The method according to claim 2, characterized in that The step of training and learning a preset gated recurrent unit model based on the training set and the test set to obtain the prediction model includes: Inputting the training set into the preset gated recurrent unit model for forward propagation to obtain a predicted value; Evaluate the preset gated recurrent unit model using a mean square error loss function according to the predicted value and the test set, and determine an update gradient of the loss function to the parameters; The preset gated recurrent unit model is back-propagated according to the updated gradient of the loss function on the parameters; the next round of training is continued until the preset gated recurrent unit model converges to obtain the prediction model.

4. The method according to claim 3, characterized in that The step of inputting the training set into the preset gated recurrent unit model for forward propagation to obtain a predicted value includes: Inputting each data sequence in the training set into the preset gated recurrent unit model, resetting each data sequence according to a reset gate of the preset gated recurrent unit model, and obtaining reset data; Use a hyperbolic tangent activation function to scale the reset data and its corresponding data sequence to obtain the state at the current moment; Using the update gate of the preset gated recurrent unit model to update the state at the current moment, to obtain a hidden state to be passed to the next node; The predicted value is obtained according to the hidden state transmitted to the next node.

5. The method according to claim 1, characterized in that The method further comprises: If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is less than the first time, evaluating the prediction model at the next preset time interval; If the accuracy of the prediction model is lower than the preset threshold and the duration of the service performance test is greater than the second time, it is determined that the prediction model cannot be evaluated.

6. The method according to any one of claims 1 to 5, characterized in that: Performance data includes service performance data and hardware performance data; The service performance data includes TPS, response time, error rate, number of concurrent connections, and number of instances; The hardware performance data includes CPU occupancy, memory occupancy, hard disk read / write rate, and network speed.

7. A device for testing and evaluating service performance stability, characterized in that: include: An acquisition unit, used to acquire actual performance data of the service performance test after the service performance test continues for a preset time and predicted performance data of the service performance test predicted by the prediction model; The prediction model is obtained by training and learning the historical performance data of the service performance stability test; a processing unit, configured to input the predicted performance data and the actual performance data into a loss function of the prediction model at a preset time interval to determine the accuracy of the prediction model; if the accuracy of the prediction model is higher than a preset threshold and the duration of the service performance test is greater than a first time, use the prediction model to predict the predicted performance data within a preset time period of the service performance test; the preset time interval is less than the first time, and the first time is less than the preset time period; Performing trend analysis based on the predicted performance data within the preset time period to obtain a trend corresponding to the predicted performance data within the preset time period; The trend corresponding to the predicted performance data within the preset time period is input into the stability risk assessment rule engine to obtain the risk of the service performance stability test assessment.

8. A computing device, characterized in that include: Memory for storing computer programs; A processor, configured to call a computer program stored in the memory, and execute the method according to any one of claims 1 to 6 according to the obtained program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the method according to any one of claims 1 to 6.

Images