Database performance prediction model training method and database performance prediction method
By adjusting and model training the historical indicators and performance data of the distributed database system, a database performance prediction model is generated, which solves the problem of low database performance prediction accuracy in the existing technology, and achieves higher prediction accuracy and reliability.
Patent Information
- Application Number
- CN202411710493.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-04
AI Technical Summary
In the prior art, the accuracy of predicting database performance of distributed database systems is mainly due to the lack of effective data-driven models relying on manual inference for early warning.
A training method for database performance prediction model is proposed. By obtaining historical indicator data and performance data, data adjustment and model training are carried out to generate a model that can predict database performance. The specific steps include obtaining the data set to be trained, adjusting the historical data, training the initial model, and obtaining the database performance prediction model.
By introducing a data-driven model training method, the prediction accuracy of database performance is improved, the dependence of manual inference is reduced, and the reliability of prediction results is enhanced.
Smart Images
Figure CN119557185A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model training and database, and in particular to a method for training a database performance prediction model and a method for predicting database performance. Background Art
[0002] A distributed database system needs to combine data from many indicators to predict the operating status of the distributed database system. The operating status can refer to the ability of the distributed database system to process data within a target time period, that is, database performance.
[0003] The prior art uses artificial reasoning to provide early warning of database performance in a distributed database system.
[0004] Therefore, there is a problem that the accuracy of predicting database performance is low. Summary of the invention
[0005] The embodiments of the present application provide a method for training a database performance prediction model, a method, device, equipment, medium and product for predicting database performance, so as to achieve the effect of improving the accuracy of predicting database performance.
[0006] In a first aspect, an embodiment of the present application provides a method for training a database performance prediction model, comprising:
[0007] Obtain a training data set of database performance indicators; wherein the training data includes historical indicator data and historical performance data, the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data;
[0008] Adjust the historical indicator data to obtain adjusted data corresponding to the historical indicator data, which is the first adjusted data; and adjust the historical performance data to obtain adjusted data corresponding to the historical performance data, which is the second adjusted data;
[0009] The preset initial model is trained according to the first adjustment data and the second adjustment data to obtain a trained database performance prediction model; wherein the database performance prediction model is used to predict performance data according to the indicator data.
[0010] In a second aspect, an embodiment of the present application provides a method for predicting database performance, including:
[0011] Get current indicator data;
[0012] The current indicator data is input into a database performance prediction model to predict performance data corresponding to the current indicator data; wherein the database performance prediction model represents the database performance prediction model of any one of claims 1-6.
[0013] In a third aspect, an embodiment of the present application provides a training device for a database performance prediction model, comprising:
[0014] An acquisition unit is used to acquire a training data set of database performance indicators; wherein the training data includes historical indicator data and historical performance data, the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data;
[0015] an adjustment unit, configured to adjust the historical indicator data to obtain adjusted data corresponding to the historical indicator data, which is the first adjusted data, and to adjust the historical performance data to obtain adjusted data corresponding to the historical performance data, which is the second adjusted data;
[0016] The training unit is used to train a preset initial model according to the first adjustment data and the second adjustment data to obtain a trained database performance prediction model; wherein the database performance prediction model is used to predict performance data according to the indicator data.
[0017] In a fourth aspect, an embodiment of the present application provides a device for predicting database performance, including:
[0018] A processing unit, used to obtain current indicator data;
[0019] The prediction unit is used to input the current indicator data into the database performance prediction model to predict the performance data corresponding to the current indicator data; wherein the database performance prediction model represents the database performance prediction model of any one of the second aspects.
[0020] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;
[0021] Memory stores computer-executable instructions;
[0022] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the first aspect and / or various possible implementations of the first aspect, the second aspect and / or various possible implementations of the second aspect as described above.
[0023] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the first aspect and / or various possible implementation methods of the first aspect, the second aspect and / or various possible implementation methods of the second aspect as described above.
[0024] In a seventh aspect, the present application provides a computer program product, comprising: a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect, the second aspect and / or various possible implementations of the second aspect.
[0025] The training method of the database performance prediction model, the prediction method, device, equipment, medium and product of database performance provided in the present application obtain a data set to be trained of database performance indicators, adjust the data set to be trained, train a preset initial model, and obtain a trained database performance prediction model; wherein the data to be trained includes historical indicator data and historical performance data, the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data; the training data set is adjusted to perform corresponding adjustment processing on the historical indicator data and the historical performance data respectively; the database performance prediction model is used to predict performance data based on the indicator data.
[0026] In addition, by obtaining current indicator data and inputting the current indicator data into a database performance prediction model, performance data corresponding to the current indicator data can be predicted.
[0027] The database performance prediction model training method and database performance prediction method provided in the present application introduce a model to train the database performance prediction model to obtain a trained database performance prediction model, so that the current indicator data can be input into the database performance prediction model to predict the performance data corresponding to the current indicator data, thereby improving the accuracy of predicting database performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0029] Figure 1 A flowchart of a method for training a database performance prediction model provided in this application;
[0030] Figure 2 A schematic diagram of a process flow of a method for predicting database performance provided in this application;
[0031] Figure 3 A schematic diagram of the prediction results of database performance provided in this application;
[0032] Figure 4 A schematic diagram of the structure of a training device for a database performance prediction model provided in this application;
[0033] Figure 5 A schematic diagram of the structure of another training device for a database performance prediction model provided in this application;
[0034] Figure 6 A schematic diagram of the structure of a database performance prediction device provided in this application;
[0035] Figure 7 A schematic diagram of the structure of the electronic device provided in this application.
[0036] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0037] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0038] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0039] A distributed database system needs to combine data from many indicators to predict the operating status of the distributed database system. The operating status can refer to the ability of the distributed database system to process data within a target time period, that is, database performance.
[0040] The prior art uses artificial reasoning to provide early warning of database performance in a distributed database system.
[0041] Therefore, there is a problem that the accuracy of predicting database performance is low.
[0042] The training method of the database performance prediction model, the prediction method, device, equipment, medium and product of database performance provided in the present application obtain a data set to be trained of database performance indicators, adjust the data set to be trained, train a preset initial model, and obtain a trained database performance prediction model; wherein the data to be trained includes historical indicator data and historical performance data, the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data; the training data set is adjusted to perform corresponding adjustment processing on the historical indicator data and the historical performance data respectively; the database performance prediction model is used to predict performance data based on the indicator data.
[0043] In addition, by obtaining current indicator data and inputting the current indicator data into a database performance prediction model, performance data corresponding to the current indicator data can be predicted.
[0044] The database performance prediction model training method and database performance prediction method provided in the present application introduce a model to train the database performance prediction model to obtain a trained database performance prediction model, so that the current indicator data can be input into the database performance prediction model to predict the performance data corresponding to the current indicator data, thereby improving the accuracy of predicting database performance.
[0045] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0046] Figure 1 The flowchart of a method for training a database performance prediction model provided in this application is shown in FIG. 1 . The execution subject of the method can be a server, a host or other device, such as Figure 1 As shown, the method may include:
[0047] S101. Obtain a training data set of database performance indicators; wherein the training data includes historical indicator data and historical performance data, the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data.
[0048] Among them, database performance indicators can refer to quantitative standards used to measure database systems.
[0049] For example, database performance indicators can help database administrators (DBAs), developers, and system analysts evaluate database performance and manage future hardware and software resources accordingly.
[0050] Exemplarily, database performance indicators may include but are not limited to response time, resource utilization, throughput, etc.
[0051] Among them, response time can refer to the time it takes from the user submitting a query request to the database returning the result;
[0052] Resource utilization can refer to CPU utilization, memory utilization, disk I / O (Input / Output) utilization, etc.
[0053] Throughput can refer to the number of transactions or queries that a database can process per unit time. The higher the throughput, the more requests the database can handle.
[0054] The data set to be trained may refer to the data set to be used for model training, and the data set to be trained includes historical indicator data and historical performance data.
[0055] The historical indicator data may refer to the database indicator data at different time points within a preset time period.
[0056] Exemplarily, the preset time period can be from January 1, 2024 to January 5, 2024, and the different time points can be any time points within the preset time period, for example, 9:00 on January 1, 2024, and the historical indicator data is the database indicator data at different time points between January 1, 2024 and January 5, 2024.
[0057] The database indicator data may be a maximum I / O usage rate of a database instance, such as 5%, 8%, 10%, etc., respectively.
[0058] Of course, the database indicator data may also be the maximum number of deletion requests of the database instance, such as 150, 120, 140, etc. in sequence.
[0059] It can be understood that the database indicator data can be obtained at different time points within a preset time period, and the interval between the time points can be 30 seconds or other numerical time intervals, which are not limited here.
[0060] Historical performance data may refer to the database operation performance corresponding to the historical indicator data.
[0061] For example, at a certain time point in history, the obtained indicator data is that the maximum I / O usage rate of the database instance is 5%, and there is performance data at this time point.
[0062] The performance data corresponding to the indicator data may be that the total number of read requests per minute in the database is 200. It can be understood that at the same time point in history, the historical indicator data and the historical performance data are in a corresponding relationship.
[0063] In a possible implementation, the method for obtaining the dataset to be trained of the database performance indicator may be to record the information of the dataset to be trained through the log function of the database, or to use a third-party tool to collect and store the dataset to be trained.
[0064] For example, the log function of PostgreSQL can record query logs, error logs, slow query logs, etc. By configuring log parameters, the information of database performance indicators can be recorded in the log file, so that the training data set of the corresponding database performance indicator can be pulled from the corresponding log file.
[0065] S102, adjusting the historical indicator data to obtain adjusted data corresponding to the historical indicator data, which is the first adjusted data, and adjusting the historical performance data to obtain adjusted data corresponding to the historical performance data, which is the second adjusted data.
[0066] Among them, the first data may refer to the data after corresponding adjustments to the historical indicator data. By adjusting the historical indicator data, noise data, outliers and inconsistent data in the historical indicator data can be eliminated, so that the subsequent training of the database performance prediction model can have higher accuracy.
[0067] The second data may refer to data adjusted accordingly to the historical performance data. By adjusting the historical performance data, noise data, outliers, and inconsistent data in the historical performance data may be eliminated, and support may be provided for subsequent training of the database performance prediction model to have higher accuracy.
[0068] In an optional implementation, the historical indicator data includes multiple indicator categories.
[0069] It can be understood that if the historical indicator data includes multiple indicator categories, the data of the multiple indicator categories can all be used as data that affects the historical performance data corresponding to the historical indicator data.
[0070] For example, if the historical indicator data is only one indicator category, that is, only the maximum I / O usage rate of the database instance, for example, 5%, 8%, 10%, etc., then the performance data corresponding to the historical indicator data is only affected by the maximum I / O usage rate of the database instance, and is not affected by indicators such as the maximum number of deletion requests of the database instance, which will lead to a poor correlation between the historical indicator data and the historical performance data.
[0071] If the historical indicator data includes multiple indicator categories, such as the maximum I / O usage of the database instance, the maximum number of deletion requests of the database instance, etc., so that the multiple indicator categories jointly affect the historical performance data, the historical indicator data and the historical performance data are highly correlated, which can improve the accuracy of the subsequent training of the database performance prediction model.
[0072] As shown in Table 1, taking the Innodb buffer pool of a distributed database as an example, database indicator data may include but is not limited to the following indicator data:
[0073] Table 1
[0074]
[0075] In an optional implementation, adjusting the historical indicator data to obtain adjusted data corresponding to the historical indicator data, which is the first adjusted data, may include:
[0076] For each indicator category, the average value of all historical indicator data corresponding to the indicator category is determined; if the difference between the historical indicator data and the average value is greater than a preset difference threshold, the historical indicator data is filtered out and the retained historical indicator data is determined as the first adjustment data.
[0077] Each indicator category may refer to any indicator category in historical indicator data.
[0078] It can be understood that for each indicator category, adjustment processing of all historical indicator data corresponding to the indicator category can be used.
[0079] The difference threshold may refer to a preset value, which is used to determine whether the difference between the historical indicator data and the average value is greater than the preset difference threshold. If it is greater than the preset difference threshold, the historical indicator data may be regarded as abnormal indicator data for screening. For example, the difference threshold is set to the standard deviation of all historical indicator data corresponding to the indicator category.
[0080] In an optional implementation, a screening formula may be used to determine the historical indicator data that needs to be screened out, and the formula satisfies:
[0081]
[0082] Among them, k can be one of the indicator categories, x i It can be the i-th historical indicator data among all the historical indicator data under the k-th indicator category, It can be the average value of all historical indicator data under the kth indicator category, σ can be the standard deviation of all historical indicator data under the kth indicator category, and the difference threshold can be set to the product of the standard deviation σ of all historical indicator data corresponding to the indicator category and the number 3.
[0083] For example, take the indicator category of the maximum number of deletion requests for a database instance as an example. For example, all the historical indicator data corresponding to the indicator category are 50, 60, 70, 80, 90, 100, 110, 120, 130, and 5000, respectively. The calculated average value of all the historical indicator data corresponding to the indicator category is 580, and the difference between the historical indicator data and the average value is 530, 520, 510, 500, 490, 480, 470, 460, 450, and 4420, respectively. The product of the standard deviation of all historical indicator data corresponding to the category and the number 3 is the difference threshold, that is, 4379. It can be seen that the difference between the data of the maximum number of deletion requests of the database instance of 5000 and the average value is greater than the difference threshold, so the maximum number of deletion requests of the database instance of 5000 is deleted, and the retained historical indicator data is the first adjustment data, that is, all historical indicator data corresponding to the indicator category are 50, 60, 70, 80, 90, 100, 110, 120, and 130 respectively.
[0084] In an optional implementation, before determining the average value of all historical indicator data corresponding to the indicator category, the following may be included:
[0085] If the historical indicator data is a null value, the historical indicator data will be filtered out.
[0086] The historical indicator data may be obtained from a log file. If the log file does not record the historical data of a certain indicator, a specific mark may be used to represent a null value. For example, the symbol representing the null value may be NULL, None, NaN (Not a Number), etc.
[0087] By filtering out historical indicator data with null values, it can provide support for subsequent training of database performance prediction models with higher accuracy.
[0088] In an optional implementation manner, determining the retained historical indicator data as the first adjustment data may include:
[0089] For each indicator category, the average value and standard deviation of the retained historical indicator data are determined; and each retained historical indicator data is updated according to the average value and the standard deviation to obtain first adjustment data.
[0090] Among them, updating can refer to converting each retained historical indicator data into data that is more in line with the model training and feature extraction, that is, first adjusting the data to ensure that the indicator data is at a suitable scale, thereby improving the effect of model training.
[0091] In a possible implementation, the update of the retained historical indicator data satisfies the formula:
[0092]
[0093] Among them, x i It can represent the historical indicator data to be updated in the retained historical indicator data, x i ′ can represent the updated historical indicator data in the retained historical indicator data, μ can represent the average value of the retained historical indicator data, and σ can represent the standard deviation of the retained historical indicator data.
[0094] Exemplarily, taking the maximum number of deletion requests for a database instance as an example, the historical indicator data retained may be 50, 60, 70, 80, 90, 100, 110, 120, and 130, respectively.
[0095] Through calculation, it can be determined that the average value of the retained historical indicator data is 90, and the standard deviation of the retained historical indicator data is 23.58.
[0096] Using the above formula that satisfies the update of the retained historical indicator data, the first adjustment data can be expressed as: -1.69, -1.27, -0.85, -0.42, 0, 0.42, 0.85, 1.27, 1.69 in sequence.
[0097] As a result, the historical indicator data is converted to a standard normal distribution with a mean of 0 and a standard deviation of 1, which helps to better train the database performance prediction model later.
[0098] In an optional implementation, other formulas may be used to determine the retained historical indicator data as the first adjustment data, such as update formula 1:
[0099]
[0100] Among them, x i It can represent the historical indicator data to be updated in the retained historical indicator data, x i ′ It can represent the updated historical indicator data in the retained historical indicator data, x min It can represent the smallest indicator data among the retained historical indicator data, x max It can represent the indicator data with the largest value among the retained historical indicator data;
[0101] For example, update formula 2:
[0102]
[0103] Among them, x i It can represent the historical indicator data to be updated in the retained historical indicator data, x i ′ It can represent updated historical indicator data among the retained historical indicator data, and N can represent a positive integer.
[0104] The updating formula or steps for determining the first adjustment data are not limited herein.
[0105] The beneficial effect of this setting is that it can eliminate noise, outliers and inconsistent data in historical indicator data, so that the subsequent training of the database performance prediction model can have higher accuracy.
[0106] In an optional implementation, adjusting the historical performance data to obtain adjusted data corresponding to the historical performance data, which is the second adjusted data, may include:
[0107] The historical performance data within a preset time period are smoothed, and the smoothed historical performance data are determined as second adjustment data.
[0108] The smoothing process may refer to converting random fluctuations and outliers in each historical performance data within a preset time period to provide a cleaner and more consistent data set, namely, the second adjustment data, for training the database performance prediction model.
[0109] The preset time period may be a pre-set time period. For example, the preset time period may be a time period corresponding to the historical indicator data and a time period of five minutes thereafter.
[0110] For example, the obtained indicator data is that the maximum I / O usage of the database instance is 5%, 8%, and 11% respectively, wherein the maximum I / O usage of the database instance is the indicator data corresponding to time point 1, time point 2, and time point 3 respectively, and the performance data corresponding to the indicator data can be that the total number of read requests of the database in the next five minutes at time point 1, time point 2, and time point 3 is 1000, 1600, and 1900. That is, the time period of the historical indicator data is the time period between time point 1 and time point 3, and the preset time period can be the time period between time point 1 and time point 3 and the time period of five minutes after time point 3.
[0111] In a possible implementation, the following formula may be used to smooth the historical performance data within a preset time period:
[0112]
[0113] Wherein, n may represent the total number of all historical performance data within a preset time period, y may represent the historical performance data to be smoothed, and u may represent the historical performance data that has been smoothed.
[0114] For example, if the historical performance data are respectively 1000, 1600, and 1900, the total number of read requests for the above-mentioned database in the next five minutes is 1000, 1600, and 1900, and the above-mentioned smoothing formula is substituted, i.e., smoothing is performed, the second adjusted data obtained can be expressed as 1050, 1500, and 1950.
[0115] It should be noted that the smoothing formula may also be in other forms, and the specific operation of the smoothing is not limited here.
[0116] The beneficial effect of such a setting is that, through smoothing, random fluctuations and outliers in historical performance data are reduced, thereby providing clearer and more stable trend data, thereby improving the accuracy of the database performance prediction model.
[0117] S103. Train a preset initial model according to the first adjustment data and the second adjustment data to obtain a trained database performance prediction model; wherein the database performance prediction model is used to predict performance data according to the indicator data.
[0118] The preset initial model may be a pre-set initial model.
[0119] In a possible implementation, the initial model is a BP (Back Propagation) neural network model, which includes an input layer, a hidden layer, and an output layer.
[0120] The input layer may include c input layer nodes, that is, the c input nodes may be used to correspond to c indicator categories of indicator data respectively, and c is a positive integer greater than or equal to 1.
[0121] It can be understood that during the training process, the first adjustment data passes through the input layer and is transmitted to the hidden layer via the input layer nodes of the input layer.
[0122] The hidden layer can refer to the middle layer between the input layer and the output layer. The hidden layer performs nonlinear transformation on the input data by applying weights and activation functions. The hidden layer adjusts the importance of each input data through weights, and introduces nonlinearity through activation functions (such as logistic function, sigmoid function, ReLU function, etc.), so that the neural network model can learn and simulate complex function mappings. Furthermore, the hidden layer can combine the various features of the input layer, extract the combined features, and pass them to the output layer.
[0123] In a possible implementation, the hidden layer may include one or more layers.
[0124] The output layer may refer to the last layer of the neural network model, which is used to generate the final prediction result, ie, the predicted value of the performance data represented by the second adjustment data.
[0125] Exemplarily, during the training process, the second adjustment data is used as the target value of the training result, which is compared with the actual output result of the output layer to calculate the error (such as mean square error, MSE), and the error network is used to backpropagate to the hidden layer to adjust the weights of each feature in the hidden layer to obtain a trained database performance prediction model.
[0126] In an optional implementation, step S103 may include:
[0127] The first adjustment data is input into a preset initial model to obtain prediction data output by the preset initial model; wherein the prediction data represents performance data predicted from historical indicator data; the similarity between the prediction data and the second adjustment data is determined, and if the similarity meets the preset training conditions, it is determined that a trained database performance prediction model is obtained.
[0128] The similarity may refer to a quantitative measure of the degree of matching between the predicted data and the actual data (ie, the second adjusted data).
[0129] Illustratively, the similarity can be calculated by a variety of statistical metrics, including but not limited to mean square error, mean absolute error (MAE), cosine similarity, Pearson correlation coefficient, etc.
[0130] The preset training condition may refer to a pre-set training condition. It can be understood that if the similarity satisfies the preset training condition, it is determined that a trained database performance prediction model is obtained.
[0131] In a possible implementation, satisfying the preset training condition may be that the similarity is greater than a similarity threshold, wherein the similarity threshold may be 97%.
[0132] If the preset training conditions are not met, the preset initial model is retrained until the similarity between the re-obtained prediction data and the second adjustment data meets the preset training conditions, and then it is determined that a trained database performance prediction model is obtained.
[0133] The beneficial effect of such a setting is that by reasonably setting the training conditions, it is possible to ensure that the database performance prediction model achieves appropriate performance during the training process, avoid overfitting and underfitting of the database performance prediction model, and improve the accuracy of the performance data predicted by the database performance prediction model.
[0134] The present application provides a method for training a database performance prediction model, which obtains first adjustment data by adjusting historical indicator data, and obtains second adjustment data by adjusting historical performance data. Based on the first adjustment data and the second adjustment data, and using a preset initial model and training conditions, an accurate database performance prediction model can be effectively trained. The model can predict the trend of future performance data based on historical indicator data, thereby improving the accuracy of the database performance prediction model in predicting performance data.
[0135] Figure 2 This is a flow chart of a method for predicting database performance provided by the present application. The execution subject of the method can be a server, a host or other device, such as Figure 2 As shown, the method may include:
[0136] S201. Obtain current indicator data.
[0137] The indicator data may refer to the database indicator data in step S101.
[0138] The current indicator data may refer to the database indicator data at the current moment.
[0139] In a possible implementation, a database monitoring tool may be used to obtain the current indicator data in real time.
[0140] Exemplarily, the current indicator data may include that the maximum I / O usage of the database instance is 5% and / or the maximum number of delete requests of the database instance is 150, etc.
[0141] It can be understood that the current indicator data may include data under at least one indicator category.
[0142] S202. Input the current indicator data into a database performance prediction model to predict performance data corresponding to the current indicator data; wherein the database performance prediction model represents a database performance prediction model of any one of the training methods of the database performance prediction model.
[0143] It can be understood that the current indicator data is used as input data of the database performance prediction model, and the output data of the database performance prediction model is the performance data corresponding to the current indicator data obtained by prediction.
[0144] Exemplarily, the performance data corresponding to the current indicator data is the total number of read requests of the database in the next five minutes, which may be 1000.
[0145] The present application provides a method for predicting database performance. By using a database performance prediction model, current indicator data is taken as input, and performance data corresponding to the current indicator data is outputted. There is no need to use a method of artificial experience prediction with poor accuracy. Instead, a database performance prediction model with high prediction accuracy is used to predict database performance, thereby improving the accuracy of predicting database performance.
[0146] In order to more intuitively describe the results of predicting database performance with high accuracy, taking the database instance Innodb buffer pool of a distributed database as an example, the predicted data for predicting database performance is the database instance Innodb bufferpool read request, that is, the database instance Innodb buffer pool read request prediction. Figure 3 A schematic diagram of the prediction results of the database performance provided in this application, such as Figure 3 As shown in the diagram of the prediction result of database performance, the horizontal axis represents time, and the vertical axis represents the total number of read requests per minute of the Innodb buffer pool of the database instance, wherein the solid line represents the actual value of the total number of read requests per minute of the Innodb buffer pool of the database instance at different time points, and the dotted line represents the predicted value of the total number of read requests per minute of the Innodb buffer pool of the database instance at different time points.
[0147] It can be seen that the actual value of the total number of read requests per minute of the Innodb buffer pool of the database instance at different time points is highly correlated with the predicted value, and the accuracy of predicting database performance is high.
[0148] Figure 4 A schematic diagram of the structure of a training device for a database performance prediction model provided in this application, such as Figure 4 As shown, the training device 40 for the database performance prediction model includes: an acquisition unit 401 , an adjustment unit 402 , and a training unit 403 .
[0149] The acquisition unit 401 is used to acquire a training data set of database performance indicators; wherein the training data includes historical indicator data and historical performance data, the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data;
[0150] The adjustment unit 402 is used to adjust the historical indicator data to obtain adjusted data corresponding to the historical indicator data, which is the first adjusted data, and to adjust the historical performance data to obtain adjusted data corresponding to the historical performance data, which is the second adjusted data;
[0151] The training unit 403 is used to train a preset initial model according to the first adjustment data and the second adjustment data to obtain a trained database performance prediction model; wherein the database performance prediction model is used to predict performance data according to the indicator data.
[0152] Figure 5 A schematic diagram of the structure of another training device for a database performance prediction model provided in this application, such as Figure 5 As shown, the training device 50 for the database performance prediction model includes: an acquisition unit 501 , an adjustment unit 502 , and a training unit 503 , wherein the adjustment unit 502 also includes a first adjustment module 5021 and a second adjustment module 5022 .
[0153] In an optional example, the historical indicator data includes multiple indicator categories, and the first adjustment module 5021 is used to determine, for each indicator category, an average value of all historical indicator data corresponding to the indicator category;
[0154] If the difference between the historical indicator data and the average value is greater than a preset difference threshold, the historical indicator data is filtered out, and the retained historical indicator data is determined as the first adjustment data.
[0155] In an optional example, before determining the average value of all historical indicator data corresponding to the indicator category, a determination module 5020 is also included for filtering out the historical indicator data if the historical indicator data is a null value.
[0156] In an optional example, the first adjustment module 5021 is further specifically configured to determine, for each indicator category, an average value and a standard deviation of the retained historical indicator data;
[0157] Each retained historical indicator data is updated according to the mean value and the standard deviation to obtain the first adjustment data.
[0158] The second adjustment module 5022 is used to smooth the historical performance data within a preset time period, and determine the smoothed historical performance data as second adjustment data.
[0159] In an optional example, the training unit 503 is further used to input the first adjustment data into a preset initial model to obtain prediction data output by the preset initial model; wherein the prediction data represents performance data predicted from historical indicator data;
[0160] The similarity between the predicted data and the second adjusted data is determined, and if the similarity meets a preset training condition, a trained database performance prediction model is determined to be obtained.
[0161] Figure 6 A schematic diagram of the structure of a database performance prediction device provided in this application, such as Figure 6 As shown, the prediction device 60 for database performance includes: a processing unit 601 and a prediction unit 602 .
[0162] Processing unit 601, used to obtain current indicator data;
[0163] The prediction unit 602 is used to input the current indicator data into the database performance prediction model to predict the performance data corresponding to the current indicator data; wherein the database performance prediction model represents a database performance prediction model of any one of the training methods of the database performance prediction model.
[0164] Figure 7 A schematic diagram of the structure of the electronic device provided in this application, such as Figure 7 As shown, the electronic device 70 provided in this embodiment includes: at least one processor 701 and a memory 702. Optionally, the device 70 further includes a communication component 703. The processor 701, the memory 702 and the communication component 703 are connected via a bus 704.
[0165] In a specific implementation process, at least one processor 701 executes the computer-executable instructions stored in the memory 702, so that at least one processor 701 executes the above method.
[0166] The specific implementation process of the processor 701 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0167] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0168] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.
[0169] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0170] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0171] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0172] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0173] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0174] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0176] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0177] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0178] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0179] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A method for training a database performance prediction model, characterized in that: include: Obtain a training data set of database performance indicators; wherein the training data includes historical indicator data and historical performance data, the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data; Adjusting the historical indicator data to obtain adjusted data corresponding to the historical indicator data as first adjusted data, and adjusting the historical performance data to obtain adjusted data corresponding to the historical performance data as second adjusted data; According to the first adjustment data and the second adjustment data, a preset initial model is trained to obtain a trained database performance prediction model; wherein the database performance prediction model is used to predict performance data according to indicator data.
2. The method according to claim 1, characterized in that The historical indicator data includes multiple indicator categories; the historical indicator data is adjusted to obtain adjusted data corresponding to the historical indicator data, which is the first adjusted data, including: For each indicator category, determining an average value of all historical indicator data corresponding to the indicator category; If the difference between the historical indicator data and the average value is greater than a preset difference threshold, the historical indicator data is filtered out, and the retained historical indicator data is determined as the first adjustment data.
3. The method according to claim 2, characterized in that Before determining the average value of all historical indicator data corresponding to the indicator category, it includes: If the historical indicator data is a null value, the historical indicator data is filtered out.
4. The method according to claim 2, characterized in that: Determining the retained historical indicator data as the first adjustment data includes: For each indicator category, determine the mean and standard deviation of the retained historical indicator data; Each of the retained historical indicator data is updated according to the average value and the standard deviation to obtain the first adjustment data.
5. The method according to claim 1, characterized in that: Adjusting the historical performance data to obtain adjusted data corresponding to the historical performance data as second adjusted data includes: Smoothing is performed on each of the historical performance data within the preset time period, and the smoothed historical performance data is determined as the second adjustment data.
6. The method according to claim 1, characterized in that According to the first adjustment data and the second adjustment data, a preset initial model is trained to obtain a trained database performance prediction model, including: Inputting the first adjustment data into the preset initial model to obtain prediction data output by the preset initial model; wherein the prediction data represents performance data predicted from historical indicator data; The similarity between the predicted data and the second adjusted data is determined, and if the similarity meets a preset training condition, a trained database performance prediction model is determined to be obtained.
7. A method for predicting database performance, characterized in that: include: Get current indicator data; The current indicator data is input into a database performance prediction model to predict performance data corresponding to the current indicator data; wherein the database performance prediction model represents the database performance prediction model described in any one of claims 1-6.
8. A training device for a database performance prediction model, characterized in that: include: An acquisition unit is used to acquire a training data set of database performance indicators; wherein the training data includes historical indicator data and historical performance data, wherein the historical indicator data represents the database indicator data at different time points within a preset time period, and the historical performance data represents the database operation performance corresponding to the historical indicator data; an adjusting unit, configured to adjust the historical indicator data to obtain adjusted data corresponding to the historical indicator data, which is first adjusted data, and to adjust the historical performance data to obtain adjusted data corresponding to the historical performance data, which is second adjusted data; A training unit is used to train a preset initial model according to the first adjustment data and the second adjustment data to obtain a trained database performance prediction model; wherein the database performance prediction model is used to predict performance data according to indicator data.
9. A device for predicting database performance, characterized in that: include: A processing unit, used to obtain current indicator data; A prediction unit, used to input the current indicator data into a database performance prediction model to predict performance data corresponding to the current indicator data; wherein the database performance prediction model represents the database performance prediction model described in any one of claims 1-6.
10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.