Database performance short-time early warning method, model training method, device and equipment

Abstract

The present specification relates to the technical field of databases, and provides a database performance short-time early warning method, a model training method, an apparatus and a device. The method comprises: obtaining historical data of influence factors affecting target database performance; generating a plurality of time series data of different time granularities by using the historical data of the influence factors; taking a deep learning model as an initial model, and training each time series data of a time granularity to obtain a to-be-candidate prediction model, thereby forming a to-be-candidate prediction model set; determining the prediction accuracy of each to-be-candidate prediction model in the to-be-candidate prediction model set; and selecting an optimal model from the to-be-candidate prediction model set as a database performance short-time prediction model according to the prediction accuracy. The embodiments of the present specification can improve the accuracy of database performance prediction.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of database, and particularly relates to a database performance short-time early warning method, a model training method, an apparatus and a device. BACKGROUND

[0002] Database performance is an important reference index for database operation. Since there is business correlation between various applications, a single database exception can cause a chain reaction and have a great business impact. Taking Mysql database as an example, Mysql database is a light, flexible and customizable database, and its size and scale are increasingly large. The Mysql performance problem that follows is undoubtedly a heavy burden for front-line operation and maintenance personnel. Therefore, performance early warning for Mysql database is particularly important.

[0003] Traditional database performance early warning is often performed when database performance is abnormal. However, at this time, the database has already had performance problems, and there is no enough time for operation and maintenance personnel to handle the exception, which directly affects the normal operation of the database in serious cases. SUMMARY

[0004] The purpose of the embodiments of the present specification is to provide a database performance short-time early warning method, a model training method, an apparatus and a device to improve the accuracy of database performance prediction.

[0005] To achieve the above purpose, on the one hand, the embodiments of the present specification provide a database performance short-time prediction model training method, comprising:

[0006] obtaining influence factor historical data affecting target database performance;

[0007] generating a plurality of time series data of different time granularities by using the influence factor historical data;

[0008] using a deep learning model as an initial model, training a to-be-candidate prediction model by using time series data of each time granularity to form a to-be-candidate prediction model set;

[0009] determining the prediction accuracy of each to-be-candidate prediction model in the to-be-candidate prediction model set;

[0010] selecting an optimal one from the to-be-candidate prediction model set as a database performance short-time prediction model according to the prediction accuracy.

[0011] In the database performance short-time prediction model training method of the embodiments of the present specification, the influence factor historical data includes historical data of internal influence factors and external influence factors.

[0012] The database performance short-time prediction model training method of the embodiments of the present specification includes:

[0013] The various candidate prediction models are sorted according to the prediction accuracy from large to small;

[0014] The difference between the first order and the second order in the sorting is determined;

[0015] It is judged whether the difference is greater than a set threshold;

[0016] If the difference is greater than the set threshold, the first order is taken as the database performance short-time prediction model.

[0017] The database performance short-time prediction model training method of the embodiments of the present specification includes:

[0018] If the difference is not greater than the set threshold, the one with longer time granularity among the first order and the second order is taken as the database performance short-time prediction model.

[0019] On the other hand, the embodiments of the present specification also provide a database performance short-time prediction model training device, which includes:

[0020] The acquisition module is configured to acquire influence factor historical data affecting target database performance;

[0021] The generation module is configured to generate time series data of multiple different time granularities by using the influence factor historical data;

[0022] The training module is configured to take a deep learning model as an initial model, and train a candidate prediction model for each time granularity by using time series data of each time granularity, thereby forming a candidate prediction model set;

[0023] The determination module is configured to determine the prediction accuracy of each candidate prediction model in the candidate prediction model set;

[0024] The selection module is configured to select an optimal one from the candidate prediction model set as a database performance short-time prediction model according to the prediction accuracy.

[0025] On the other hand, the embodiments of the present specification also provide a database performance short-time early warning method, which includes:

[0026] According to a specified time granularity, influence factor data affecting target database performance is collected to form time series data of the specified time granularity;

[0027] inputting the time series data of the specified time granularity into a database performance short-time prediction model to obtain performance parameters of the target database within a prediction step range; the database performance short-time prediction model is an optimal one of a plurality of candidate prediction models, the plurality of candidate prediction models being obtained by training a deep learning model using time series data of different time granularities;

[0028] deciding whether to output database performance early warning information according to the performance parameters of the target database within the prediction step range.

[0029] In the database performance short-time early warning method of the embodiments of the present disclosure, the historical data of the influence factors includes historical data of internal influence factors and external influence factors.

[0030] In another aspect, the embodiments of the present disclosure also provide a database performance short-time early warning device, which comprises:

[0031] The collection module is configured to collect influence factor data affecting performance of a target database according to a specified time granularity to form time series data of the specified time granularity.

[0032] The prediction module is configured to input the time series data of the specified time granularity into a database performance short-time prediction model to obtain performance parameters of the target database within a prediction step range; the database performance short-time prediction model is an optimal one of a plurality of candidate prediction models, the plurality of candidate prediction models being obtained by training a deep learning model using time series data of different time granularities.

[0033] The decision module is configured to decide whether to output database performance early warning information according to the performance parameters of the target database within the prediction step range.

[0034] In another aspect, the embodiments of the present disclosure also provide a computer device, which comprises a memory, a processor, and a computer program stored in the memory, and when the computer program is run by the processor, instructions of the above method are executed.

[0035] In another aspect, the embodiments of the present disclosure also provide a computer storage medium, which stores a computer program, and when the computer program is run by a processor of a computer device, instructions of the above method are executed.

[0036] In another aspect, the embodiments of the present disclosure also provide a computer program product, which comprises a computer program, and when the computer program is run by a processor of a computer device, instructions of the above method are executed.

[0037] It can be seen from the technical solutions provided by the embodiments of the present specification that the database performance short-time prediction model of the embodiments of the present specification is the optimal one selected from a plurality of candidate prediction models trained by using time series data of different time granularities, so that the accuracy of the short-time performance prediction of the database can be improved. The accurate short-time prediction can gain time for the database abnormality response to a certain extent, and thus is beneficial to maintaining the running stability of the database. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments described in the present specification, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0039] Figure 1 A structural schematic diagram of a database performance short-time early warning system in some embodiments of the present specification is shown;

[0040] Figure 2 A flowchart of a database performance short-time prediction model training method in some embodiments of the present specification is shown;

[0041] Figure 3 A flowchart of a database performance short-time early warning method in some embodiments of the present specification is shown;

[0042] Figure 4 A structural schematic diagram of a database performance short-time prediction model in some embodiments of the present specification is shown;

[0043] Figure 5 A structural block diagram of a database performance short-time prediction model training device in some embodiments of the present specification is shown;

[0044] Figure 6 A structural block diagram of a database performance short-time early warning device in some embodiments of the present specification is shown;

[0045] Figure 7 A structural block diagram of a computer device in some embodiments of the present specification is shown.

[0046]

Explanation of reference numerals

[0047] 100, database;

[0048] 200, database performance short-time early warning device;

[0049] 51, acquisition module;

[0050] 52, generation module;

[0051] 53. a training module;

[0052] 54. a determination module;

[0053] 55. a selection module;

[0054] 61. a collection module;

[0055] 62. a prediction module;

[0056] 63. a decision module;

[0057] 702. a computer device;

[0058] 704. a processor;

[0059] 706. a memory;

[0060] 708. a driving mechanism;

[0061] 710. an input / output interface;

[0062] 712. an input device;

[0063] 714. an output device;

[0064] 716. a presentation device;

[0065] 718. a graphical user interface;

[0066] 720. a network interface;

[0067] 722. a communication link;

[0068] 724. a communication bus. DETAILED DESCRIPTION

[0069] In order to make the technical personnel in the technical field better understand the technical solutions in the specification, the technical solutions in the specification will be described clearly and completely in the following by combining the drawings in the specification of the embodiment. Obviously, the described embodiments are only part of the embodiments of the specification, not all. Based on the embodiments in the specification, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the scope of protection of the specification.

[0070] Studies have shown that before the performance of a database is abnormal, there will be abnormal fluctuations in some performance indicators. If the database performance problem can be found in advance and solved or avoided, the stability of the system and the business continuity to the outside can be improved. However, the database performance has many influencing factors, is complex and highly nonlinear. Although the database performance fluctuation has its own periodicity and regularity, many uncertain factors will cause the database performance to fluctuate. At the same time, the database performance has a strong nonlinear characteristic, and with the shortening of the prediction time, this uncertainty and nonlinearity may be more significant. However, the increase of the prediction time will significantly affect the accuracy of the prediction, so it is extremely challenging to accurately predict the database performance.

[0071] Therefore, the embodiments of the present specification provide a database short-time performance prediction scheme to improve the accuracy of database performance prediction. Among them, the database short-time performance prediction refers to predicting the performance of the database at t+Δt time at t time.

[0072] Reference Figure 1 As shown in the figure, the database short-time performance prediction system of the embodiments of the present specification can include a database 100 and a database short-time performance early warning device 200. The database short-time performance early warning device 200 is configured with a pre-trained database performance short-time prediction model. The database performance short-time prediction model is the best one selected from a plurality of candidate prediction models. The plurality of candidate prediction models are all obtained by using different time granularity time series data (a kind of influence factor historical data) to train the initial model of a deep learning model.

[0073] When performing database short-time performance prediction, the database short-time performance early warning device 200 can collect influence factor data affecting the database 100 from the database 100, extract time series data of a specified time granularity using the influence factor data, and input the time series data of the specified time granularity into the database performance short-time prediction model. The performance parameters of the database 100 within the prediction step range are predicted, so as to decide whether to output the database performance early warning information according to the performance parameters. Since the database performance short-time prediction model is the best one selected from a plurality of candidate prediction models trained by using time series data of different time granularities, the accuracy of the database short-time performance prediction can be improved. This accurate short-time prediction in advance can gain time for the database abnormality response processing to some extent, thereby being beneficial to maintaining the running stability of the database.

[0074] For ease of understanding, before describing the database performance short-time early warning method of the embodiments of the present specification, the training process of the database performance short-time prediction model is described. Reference Figure 2 As shown in the figure, in some embodiments, the database performance short-time prediction model training method can include the following steps:

[0075] Step 201, acquire historical data of influencing factors affecting the performance of the target database.

[0076] Before acquiring the historical data of influencing factors affecting the performance of the target database, a target database performance prediction index system is constructed in advance. The target database performance prediction index system can include input indicators and prediction indicators. The input indicator data can be used as input in the model training process, and the prediction indicator data is the output data of the model.

[0077] Taking a Mysql database as an example, there are many factors affecting the performance of the Mysql database. The traditional analysis of the influencing factors affecting the performance of the Mysql database generally only considers internal influencing factors, while the embodiments of the present specification comprehensively consider internal factors and external factors affecting the performance of the Mysql database, so as to further improve the accuracy of short-term prediction of database performance.

[0078] The internal factors affecting the performance of the Mysql database mainly include server software and hardware factor indicators and database busy degree indicators of the Mysql database. In some embodiments, the software and hardware factor indicators may, for example, include server CPU, memory, disk size, network card version, operating system version, server concurrency, whether to enable hyper-threading, whether to install middleware, etc. In some embodiments, the database busy degree indicators may, for example, include whether it is in a business peak period, database connection number, thread pool accumulation degree, number of lock waiting in the database, CPU usage, memory usage, etc.

[0079] The internal factors affecting the performance of the Mysql database may include: server service life, computer room temperature, computer room humidity, whether it is in a business peak period, etc.

[0080] The historical data corresponding to the internal factors and external factors affecting the performance of the Mysql database form the historical data of influencing factors together. In some embodiments, the output of model training (i.e., the prediction indicator) can use CPU usage, memory usage, etc.

[0081] In some embodiments, the target database performance prediction index system can be as shown in Table 1.

[0082] Table 1

[0083]

[0084] On the basis of determining the target database performance prediction index system, the influence factor historical data of the target database can be selected for model training. The influence factor historical data is the influence factor data in a specified historical time range. For example, the influence factor historical data of the target database in the last 5 days. The influence factor historical data obtained from the database often contains dirty data, so data preprocessing is needed before using these data.

[0085] In some embodiments, data preprocessing can include:

[0086] (1) Abnormal data correction

[0087] ① Correction of redundant and repetitive data

[0088] For redundant and repetitive data, if the records are completely repeated, only one record is retained; if they are partially repeated, they are analyzed according to the historical performance records of the Mysql database and combined into one record.

[0089] ② Correction of irrelevant data

[0090] Irrelevant data refers to data that has no impact on Mysql performance analysis and prediction, which can be directly deleted.

[0091] ③ Correction of missing and abnormal data

[0092] Correction of missing and abnormal data is an important part of data correction. If the data is missing or abnormal, it can be supplemented or corrected according to the performance of the database on the same day. If the missing or abnormal situation is serious and cannot be supplemented or corrected according to the performance of the database on the same day, it is deleted.

[0093] (2) Quantitative processing of non-quantitative data

[0094] In addition, CPU size, memory size, disk size, server concurrency, database connection number, thread pool accumulation degree, database lock waiting number, CPU usage, memory usage, server usage time are quantitative data, while network card version, operating system version, whether to enable hyper-threading, whether to install middleware, whether in the business peak period are non-quantitative data, which need to be quantitatively converted. The quantitative conversion method used in this application is: if there are n types of network card versions in the sample, the quantized data is 1, 2, …, n, and the operating system version quantization method is the same as the network card version; if the server enables hyper-threading, it is quantized as 1, and if it does not, it is quantized as 0, and the quantization method of whether to install middleware and whether in the business peak period is the same as whether to enable hyper-threading.

[0095] Step 202, generating a plurality of time series data of different time granularities by using the influence factor historical data.

[0096] As described above, the database performance has strong nonlinear characteristics, and as the prediction time is shortened, the uncertainty and nonlinearity can be more significant; and the increase of the prediction time can significantly affect the accuracy of the prediction; and the reasonable selection of the time granularity is extremely important for improving the prediction accuracy of the model. Therefore, it is necessary to generate a plurality of time series data of different time granularities by using the influence factor historical data.

[0097] In the embodiments of the present specification, the time granularity is the sampling time interval, and the time granularity is an integer multiple of the original sampling time interval of the influence factor historical data; for example, the original sampling time interval is 1 second, and one time granularity is 5 minutes, and the time granularity is 300 times of the original sampling time interval.

[0098] Therefore, when the time granularity is 5 minutes, the influence factor historical data is resampled with a sampling time interval of 5 minutes to obtain one time series data; when the time granularity is 10 minutes, the influence factor historical data is resampled with a sampling time interval of 10 minutes to obtain another time series data; when the time granularity is 15 minutes, the influence factor historical data is resampled with a sampling time interval of 15 minutes to obtain another time series data; in this way, when the influence factor historical data is resampled by using a plurality of time granularities, a plurality of time series data of different time granularities can be formed.

[0099] In an exemplary embodiment, taking 5-minute time granularity as an example, for any Mysql database server, the time series is composed of: X=(X1, X2,..., X t ,...,X n ), Y=(Y1, Y2,..., Y t ,...,Y n ), each X t , Y t is a two-dimensional vector, X t represents the values of 15 internal and external influence factor indicators of a server at time t (sampling time), and Y t represents the values of 2 predicted indicators of a server at time t (sampling time). That is, at time t, the values of each internal and external influence factor X t are: CPU size, X t1 ; memory size, X t2 ; disk size: X t3 ; network card version: X t4 ; operating system version: X t5 ; server concurrency: X t6; whether to open hyperthreading: X t7 ; whether to install middleware: X t8 ; whether it is in the peak period of business: X t9 ; database connection number: X t10 ; thread pool accumulation degree: X t11 ; the number of lock waiting in the database: X t12 ; cpu usage: X t13 ; memory usage X t14 ; server usage time: X t15 ; temperature of machine room: X t16 ; humidity of machine room: X t17 . Y t represents the value of the predicted index of a server at t+1: cpu usage: Y t1 ; memory usage: Y t2 When the time granularity is 10 minutes, X=(X1, X2,…, X t , …, X n / 2 ), Y=(Y1, Y 32 ,…, Y t , …, Y n / 2 ); when the time granularity is 15 minutes, X=(X1, X2,…, X t , …, X n / 3 ), Y=(Y1, Y 32 ,…, Y t , …, Y n / 3 ).

[0100] Step 203, using the deep learning model as the initial model, training each time granularity time series data to obtain a candidate prediction model, forming a candidate prediction model set.

[0101] Compared with other machine learning initial models, using the deep learning model as the initial model can improve the accuracy of the model and shorten the model training period.

[0102] After obtaining the time series data of each time granularity, all of them can be taken as a dataset or a part of them can be selected as a dataset according to needs. For example, a dataset can be selected according to a time step. The time step represents the data range (i.e., the number of data points) selected from the time series data. For example, taking the time series data of a 5-minute time granularity as an example, if the time step is 3, the historical data of the last sampling time (for example, 2021-01-01 10:00:00), the time 5 minutes ago (for example, 2021-01-01 09:55:00), and the time 10 minutes ago (for example, 2021-01-01 09:50:00) in the time series data of the 5-minute time granularity can be taken as a dataset. If the time step is 4, the historical data of the initial sampling time, the time 5 minutes ago, the time 10 minutes ago, and the time 15 minutes ago in the time series data of the 5-minute time granularity can be taken as a dataset, and so on.

[0103] Before training, a random sampling method can be used to divide the time series data of each time granularity into datasets (for example, into a training set and a test set) according to a set proportion.

[0104] For the training set and the test set corresponding to the time series data of each time granularity, the same initial model can be used for model training. Therefore, when there are N time series data of time granularity, N initial models are needed.

[0105] Reference Figure 4 As shown in FIG. 1, in some embodiments, the deep learning model can include an input layer, a BN network layer, an LSTM network layer, a fully connected network layer, an inverse BN network layer, and an output layer. The following will be described respectively.

[0106] (I) Input layer: training set input.

[0107] (II) BN network: Since the collected original data are all data with dimensions, batch normalization (BN) processing is performed in the input layer to convert the collected data with dimensions into data without dimensions through BN normalization processing, so as to facilitate subsequent calculation. The BN normalization can reduce the requirement for model learning rate to a certain extent, accelerate the convergence speed of the model, destroy the original data distribution, alleviate overfitting to a certain extent, and thus improve the training accuracy of the model.

[0108] BN normalization process:

[0109]

[0110]

[0111] xi For the i-th sample data, m is the number of samples (i.e., batch size), μ B is the sample mean, is the sample variance.

[0112] After BN normalization, we get:

[0113]

[0114] where x′ i is x i After BN normalization, ε is a constant value (very small) added to avoid the denominator being zero.

[0115] The data after the above BN normalization is reconstructed, that is, a linear transformation operation is added to make the data as much as possible to restore its own expression ability. Where γ, β are variable parameters, the initial values are 1 and 0 respectively, and can be adjusted to the appropriate value through learning.

[0116] X′ i = γx′ i + β

[0117] (Three) Long Short-Term Memory Network (LSTM) layer: adjust the neuron state through the input gate, the forgetting gate and the output gate.

[0118] LSTM is improved on the basis of Recurrent Neural Network (RNN), which not only maintains the excellent performance of RNN, but also improves the problem of gradient disappearance. And the "time memory unit" contained in LSTM can effectively learn the long and short term dependence information of time series, so it is more suitable for processing and prediction of time series interval and delay problems.

[0119] The forgetting gate defines the information that needs to be forgotten by the neuron. Please continue to refer to Figure 4 , the forgetting gate can read the input x t at the current time, and can also read the state h t-1 of the hidden layer at the last time, and finally output a value between 0 and 1 to C t-1 . Among them, 1 represents "completely retain" the information, and 0 represents "completely discard" the information. The calculation formula is as follows:

[0120] f t = σ(W f · [h t-1 , x t ] + b f )

[0121] The input gate defines new information to be stored in the neuron (Cell) state. The input gate consists of two parts, one of which defines a new candidate value vector through a tanh layer, and the other defines the value to be input through a sigmoid layer. The new candidate value vector will be added to the new Cell state C t . The calculation formula is as follows:

[0122] i t = σ(W i · [h t-1 , x t ] + b i )

[0123] At this time, a new contribution will be made to the neuron state, and the formula is as follows:

[0124]

[0125] At the same time, the input gate will update the Cell state, updating the original state C t-1 to C t . The calculation formula is as follows, where the information to be discarded is f t · C t-1 .

[0126]

[0127] The output gate defines the value to be output. The output is based on the Cell state, first, the sigmoid function is used to determine the information of the Cell state that needs to be output, and then the Cell state is processed through the tanh layer (so that the output value is between [-1, 1]) and multiplied by the output of the sigmoid gate, so that the final output is determined. The calculation formula is as follows:

[0128] o t = σ(W o · [h t-1 , x t ] + b o )

[0129] h t = o t · tanh(C t )

[0130] Where f, i, C, o represent the forget gate, the input gate, the Cell state, and the output gate, respectively, W is the corresponding weight, b is the corresponding bias (intercept), σ is the sigmoid function, tanh is the hyperbolic tangent activation function, and x t is the input vector of the model at time t.

[0131] (iv) Fully connected network layer: i.e. Dense layer. The role of this layer is to convert the output of the LSTM into a dimension, and obtain the predicted value of the model. After obtaining the predicted value, the error between the true value and the predicted value is calculated, so as to adjust the weight value between each layer of the network.

[0132] (v) Anti-BN network:

[0133] The predicted value After the anti-batch normalization processing, the is obtained as the output of the model.

[0134] (vi) Output layer: predicted value output.

[0135] Step 204, determining the prediction accuracy of each candidate prediction model in the set of candidate prediction models.

[0136] The same evaluation system can be used for evaluation, so as to compare the advantages and disadvantages between each candidate prediction model trained based on time series data of each time granularity.

[0137] For example, in some embodiments, the prediction accuracy of each candidate prediction model in the set of candidate prediction models can be evaluated by root mean square error (RMSE) and mean absolute error (MAE). For example, if the predicted value is (normalized value), and the true value is (normalized value), the smaller the calculated RMSE and MAE values, the closer the predicted value of the established short-term MYSQL performance prediction model to the true value, i.e. the better the model prediction performance.

[0138]

[0139]

[0140] Step 205, selecting the optimal one from the set of candidate prediction models as the database performance short-term prediction model according to the prediction accuracy.

[0141] In some embodiments, selecting the optimal one from the set of candidate prediction models as the database performance short-term prediction model according to the prediction accuracy can include:

[0142] (1) sorting the candidate prediction models in descending order according to the prediction accuracy.

[0143] (2) determining the difference between the first and second order in the sorting.

[0144] (3) Determine whether the difference is greater than a set threshold. If the difference is greater than the set threshold, it indicates that the prediction accuracy of the first-ranked model is significantly greater than that of the others in the ranking. Therefore, the first-ranked model can be used as a short-term prediction model for database performance. If the difference is not greater than the set threshold, it indicates that the difference between the first-ranked model and the second-ranked model is extremely small (to the point that it can be ignored). In this case, the model with the longer time granularity between the first-ranked model and the second-ranked model can be used as a short-term prediction model for database performance. In this way, accurate and longer-term prediction data can be obtained, thereby allowing more emergency time for subsequent anomaly response when the database is abnormal.

[0145] Thus, by comparing the prediction accuracy of models trained at different time granularities, the time granularity corresponding to the best prediction accuracy and the corresponding model can be determined.

[0146] Based on the aforementioned short-term database performance prediction model, this model can be used for short-term prediction (early warning) of real-time database performance. Therefore, this specification also provides a short-term database performance early warning method that can be applied to… Figure 1 In the database performance short-term early warning device. (Reference) Figure 3 As shown, in some embodiments, the database performance short-term warning method may include the following steps:

[0147] Step 301: Collect data on factors affecting the performance of the target database according to the specified time granularity, and form time series data at the specified time granularity.

[0148] The specified time granularity is the same as the time granularity corresponding to the short-term database performance prediction model. For example, if the time granularity corresponding to the short-term database performance prediction model is 5 minutes, then the specified time granularity is also 5 minutes.

[0149] Step 302: Input the time series data at the specified time granularity into the database performance short-term prediction model to predict the performance parameters of the target database within the prediction step size range.

[0150] Step 303: Based on the performance parameters of the target database within the prediction step range, decide whether to output database performance warning information.

[0151] The prediction step size is the prediction time span (in the embodiments of this specification, the prediction step size generally does not exceed 30 minutes). For example, if the prediction step size is Δt and the current time is t, then the database performance parameters within the future range of t+Δt can be predicted at time t. For example, if the prediction step sizes are 5 minutes, 10 minutes, and 15 minutes, then the database performance fluctuations within the next 5 minutes, 15 minutes, and 30 minutes from the current time can be predicted.

[0152] For example, the performance parameters of the target database in the prediction step range can be compared with the corresponding normal range, if the performance parameters in the prediction step range exceed the normal range, a pre-warning information can be output to remind the relevant personnel to deal with; if the performance parameters in the prediction step range do not exceed the normal range, no pre-warning information is output. Since the database performance short-time prediction model can perform short-time and accurate database performance short-time prediction, the database abnormality can be accurately predicted based on the database performance short-time prediction model, that is, false alarm is reduced or avoided, and missed alarm is also reduced or avoided.

[0153] Although the process flow described above includes a plurality of operations appearing in a specific order, it should be clearly understood that the processes can include more or fewer operations, which can be executed sequentially or in parallel (for example, using parallel processors or multi-threaded environments).

[0154] Corresponding to the database performance short-time prediction model training method shown in the above Figure 2 The present embodiment also provides a database performance short-time prediction model training device. Referring to the above Figure 5 In some embodiments, the database performance short-time prediction model training device can include:

[0155] The acquisition module 51 is configured to acquire influence factor historical data affecting the performance of a target database;

[0156] The generation module 52 is configured to generate a plurality of time series data of different time granularities using the influence factor historical data;

[0157] The training module 53 is configured to use each time series data of a time granularity to train a to-be-candidate prediction model respectively using a deep learning model as an initial model, to form a to-be-candidate prediction model set;

[0158] The determination module 54 is configured to determine the prediction accuracy of each to-be-candidate prediction model in the to-be-candidate prediction model set;

[0159] The selection module 55 is configured to select an optimal one from the to-be-candidate prediction model set as a database performance short-time prediction model according to the prediction accuracy.

[0160] Corresponding to the database performance short-time pre-warning method shown in the above Figure 3 The present embodiment also provides a database performance short-time pre-warning device. Referring to the above Figure 6 In some embodiments, the database performance short-time pre-warning device can include:

[0161] The acquisition module 61 is configured to acquire influence factor data affecting the performance of a target database according to a specified time granularity, to form time series data of the specified time granularity;

[0162] a prediction module 62, configured to input the time series data of the specified time granularity into a database performance short-time prediction model to obtain performance parameters of the target database in a prediction step range, wherein the database performance short-time prediction model is an optimal one from a plurality of candidate prediction models, and the plurality of candidate prediction models are obtained by training a deep learning model using time series data of different time granularities.

[0163] a decision module 63, configured to determine whether to output database performance warning information according to the performance parameters of the target database in the prediction step range.

[0164] For ease of description, the above apparatus is described in various units in terms of functions. Of course, the functions of the units can be implemented in one or more software and / or hardware in the implementation of the present specification.

[0165] It should be noted that in the embodiments of the present specification, the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user and authorized by all parties.

[0166] The embodiments of the present specification also provide a computer device. As shown in Figure 7 In some embodiments of the present specification, the computer device 702 can include one or more processors 704, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each of which can implement one or more hardware threads. The computer device 702 can also include any memory 706 for storing any kind of information such as code, settings, data, etc. In a specific embodiment, the memory 706 stores a computer program executable on the processor 704, and when the computer program is executed by the processor 704, the computer program can perform the above Figure 2 and Figure 3The instructions of the method described. Without limitation, for example, the memory 706 can include any one or combination of the following: any type of RAM, any type of ROM, a flash memory device, a hard disk, an optical disk, and so on. More generally, any memory can use any technology for storing information. Further, any memory can provide volatile or nonvolatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 702. In one case, the computer device 702 can perform any of the operations described by associated instructions stored in any memory or combination of memories that are processed by the processor 704. The computer device 702 also includes one or more drive mechanisms 708, such as a hard drive mechanism, an optical disk drive mechanism, and so on, for interacting with any memory.

[0167] The computer device 702 can also include an input / output interface 710 (I / O) for receiving various input (via input devices 712) and for providing various output (via output devices 714). One particular output mechanism can include a presentation device 716 and an associated graphical user interface 718 (GUI). In other embodiments, the input / output interface 710 (I / O), the input devices 712, and the output devices 714 can also not be included, just as a computer device in a network. The computer device 702 can also include one or more network interfaces 720 for exchanging data with other devices via one or more communication links 722. One or more communication buses 724 couple the above-described components together.

[0168] The communication links 722 can be implemented in any manner, such as through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, and so on, or any combination thereof. The communication links 722 can include any combination of hardwired links, wireless links, routers, gateway functionality, name servers, and so on governed by any protocol or combination of protocols.

[0169] The present application is described with reference to the drawings using a flowchart and / or a block diagram of methods, apparatus (systems) and computer program products according to some embodiments of the present application. It will be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 The flowchart and / or block diagram can include one or more flowcharts and / or one or more block diagrams. Figure 1 The flowchart and / or block diagram can include one or more flowcharts and / or one or more block diagrams.

[0170] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0172] In a typical configuration, a computer device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0173] The memory can include non-persistent memory and / or volatile memory, e.g., random access memory (RAM) and / or non-volatile memory, e.g., read-only memory (ROM) or flash memory. The memory is an example of computer-readable media.

[0174] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic disks storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computer device. According to the definition in this specification, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

[0175] Those skilled in the art will appreciate that embodiments of the present specification can be readily used as a method, a system or a computer program product. Accordingly, embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present specification can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code thereon for use by or in connection with an instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. The computer-usable or computer readable program code can be downloaded from an Internet website, server, or other remote source.

[0176] Embodiments of the present specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Embodiments of the present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0177] It should also be understood that, in the embodiments of the present specification, the term "and / or" merely describes an associated relationship between associated objects, and means that three relationships can exist. For example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally means that the front and rear associated objects are in an "or" relationship.

[0178] Various embodiments of the present specification are described in a progressive manner, and the same or similar parts among the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0179] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present specification. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0180] The above merely provides an example of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. A method for training a short-term database performance prediction model, characterized in that, include: Obtain historical data on factors affecting the performance of the target database; The historical data of the influencing factors includes historical data of both internal and external influencing factors. The internal influencing factors include hardware and software factors and database activity levels. Hardware and software factors include CPU, memory, disk size, network card version, operating system version, server concurrency, whether hyper-threading is enabled, and whether middleware is installed. Database activity levels include whether it is during peak business hours, number of database connections, thread pool backlog, number of lock waits in the database, CPU utilization, and memory utilization. External influencing factors include the server's age, data center temperature, and data center humidity. Multiple time series data with different time granularities are generated using the historical data of the aforementioned influencing factors; Using a deep learning model as the initial model, a candidate prediction model is trained using time series data at each time granularity, forming a set of candidate prediction models. Determine the prediction accuracy of each candidate prediction model in the candidate prediction model set; Based on the prediction accuracy, the best candidate prediction model is selected from the set of candidate prediction models as the short-term prediction model for database performance. The step of selecting the optimal candidate prediction model from the set of candidate prediction models based on the prediction accuracy as the short-term prediction model for database performance includes: The candidate prediction models are sorted from highest to lowest according to their prediction accuracy. Determine the difference between the first and second items in the sorting; Determine whether the difference is greater than a set threshold; If the difference is greater than a set threshold, the first in line will be used as the short-term prediction model for database performance. If the difference is not greater than a set threshold, then the one with the longer time granularity between the first and second priorities will be used as the short-term prediction model for database performance.

2. A training device for a short-term prediction model of database performance, characterized in that, include: The acquisition module is used to acquire historical data on factors that affect the performance of the target database. The historical data of the influencing factors includes historical data of both internal and external influencing factors. The internal influencing factors include hardware and software factors and database activity levels. Hardware and software factors include CPU, memory, disk size, network card version, operating system version, server concurrency, whether hyper-threading is enabled, and whether middleware is installed. Database activity levels include whether it is during peak business hours, number of database connections, thread pool backlog, number of lock waits in the database, CPU utilization, and memory utilization. External influencing factors include the server's age, data center temperature, and data center humidity. The generation module is used to generate multiple time series data with different time granularities using the historical data of the influencing factors; The training module is used to train a candidate prediction model using a deep learning model as the initial model and time series data at each time granularity to form a set of candidate prediction models. The determination module is used to determine the prediction accuracy of each candidate prediction model in the candidate prediction model set. The selection module is used to select the best one from the set of candidate prediction models as the short-term prediction model for database performance based on the prediction accuracy. Specifically, the selection module is used for: The candidate prediction models are sorted from highest to lowest according to their prediction accuracy. Determine the difference between the first and second items in the sorting; Determine whether the difference is greater than a set threshold; If the difference is greater than a set threshold, the first in line will be used as the short-term prediction model for database performance. If the difference is not greater than a set threshold, then the one with the longer time granularity between the first and second priorities will be used as the short-term prediction model for database performance.

3. A short-term early warning method for database performance, characterized in that, include: Collect data on factors affecting the performance of the target database at a specified time granularity, and form time series data at the specified time granularity; The time series data at the specified time granularity is input into the database performance short-term prediction model to predict the performance parameters of the target database within the prediction step size range; the database performance short-term prediction model is the best among multiple candidate prediction models, which are trained using a deep learning model as the initial model and time series data at different time granularities. The database performance short-term prediction model is trained using the database performance short-term prediction model training method described in claim 1; The decision on whether to output database performance warning information is based on the performance parameters of the target database within the prediction step size range.

4. A short-term early warning device for database performance, characterized in that, include: The data acquisition module is used to collect data on factors affecting the performance of the target database at a specified time granularity, and form time series data at the specified time granularity. The prediction module is used to input the time series data at the specified time granularity into the database performance short-term prediction model to predict the performance parameters of the target database within the prediction step size range; the database performance short-term prediction model is the best among multiple candidate prediction models, and the multiple candidate prediction models are trained using a deep learning model as the initial model and time series data at different time granularities. The database performance short-term prediction model is trained using the database performance short-term prediction model training method described in claim 1; The decision module is used to decide whether to output database performance warning information based on the performance parameters of the target database within the prediction step size range.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it executes the instructions of the method according to any one of claims 1 and 3.

6. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor of the computer device, it executes the instructions of the method according to any one of claims 1 and 3.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, performs instructions according to the method of any one of claims 1 and 3.

Images