Database performance prediction method and device, equipment and medium

Abstract

The invention provides a database performance prediction method which can be applied to the field of big data or cloud computing. The method comprises the steps of obtaining performance data of a target database in a historical time period; constructing an initial feature matrix with time features based on the performance data; inputting the initial feature matrix into a deep neural network in a pre-trained regression prediction model, and extracting performance features of the performance data; and based on the performance characteristics, predicting the change degree of the performance data in the future time period by using a regression prediction layer of the regression prediction model so as to carry out early warning on the target database. The invention further provides a database performance prediction device and equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of big data or cloud computing, and more specifically to a database performance prediction method, device, equipment, medium and program product. Background Art

[0002] Daily operation and maintenance of databases is one of the important tasks of data centers. The core goal is to ensure the healthy and smooth operation of the database and ensure the integrity, security and availability of the data. As a database operation and maintenance personnel, you can judge whether the operation status of a database is healthy and normal from the various performance indicators of a database. However, in the field of deep learning, there are relatively few studies on database performance prediction, and the existing performance prediction models show poor adaptability when facing different database loads, hardware configurations and network environments. At the same time, due to the complexity of database performance, the existing prediction models may not be able to accurately capture the changing trends of database performance, resulting in a large deviation between the prediction results and the actual values. In addition, the prediction model can usually only make predictions for specific database scenarios, and it is difficult to adapt to different business needs and changes. Summary of the invention

[0003] In view of the above problems, embodiments of the present disclosure provide a method, apparatus, device, medium and program product for predicting performance of a database.

[0004] According to a first aspect of the present disclosure, a method for predicting performance of a database is provided, the method comprising: obtaining performance data of a target database in a historical time period; constructing an initial feature matrix with time characteristics based on the performance data; inputting the initial feature matrix into a deep neural network in a pre-trained regression prediction model to extract performance characteristics of the performance data; and based on the performance characteristics, using the regression prediction layer of the regression prediction model to predict the degree of change of the performance data in a future time period, so as to issue an early warning to the target database.

[0005] According to an embodiment of the present disclosure, obtaining the performance data of the target database for a historical time period includes: identifying the main database of the target database; and classifying each of the main databases according to identification information of the main database, and obtaining the performance data of the main database corresponding to the classification, wherein the identification information is used to characterize the attributes of each of the main databases.

[0006] According to an embodiment of the present disclosure, the method also includes: integrating the performance data to obtain a performance data set, and determining different density thresholds based on the performance data set; detecting the performance data set based on the different density thresholds to identify outliers in the performance data set; and eliminating outliers in the performance data set to obtain the remaining performance data of the performance data set.

[0007] According to an embodiment of the present disclosure, the method also includes: identifying missing values ​​of the performance data set and filling the missing values ​​using a mean filling method; and normalizing the performance data set with the missing values ​​filled using a maximum-minimum normalization method.

[0008] According to an embodiment of the present disclosure, training the deep neural network includes: inputting the initial feature matrix into the deep neural network to obtain the output of the deep neural network; and when the difference between the output of the deep neural network and the true feature representation is greater than a first specified threshold, adjusting the parameters and structure of the deep neural network until the difference between the output of the deep neural network and the true feature representation is less than or equal to the first specified threshold.

[0009] According to an embodiment of the present disclosure, the method further includes: integrating a regression prediction layer based on the trained deep neural network to construct the regression prediction model.

[0010] According to an embodiment of the present disclosure, training the regression prediction model includes: inputting the performance characteristics output by the deep neural network into the regression prediction layer to obtain the degree of change of the performance data output by the regression prediction layer; using a loss function to calculate the difference between the statistical distribution of the predicted degree of change and the statistical distribution of the actual change; and when the difference between the statistical distribution of the degree of change and the statistical distribution of the actual change is greater than a second specified threshold, updating the parameters of the regression prediction model through an optimizer according to the gradient of the loss function until the difference is less than or equal to the second specified threshold.

[0011] A second aspect of the present disclosure provides a performance prediction device for a database, the device comprising: an acquisition module for acquiring performance data of a target database in a historical time period; a construction module for constructing an initial feature matrix with time characteristics based on the performance data; an extraction module for inputting the initial feature matrix into a deep neural network in a pre-trained regression prediction model to extract performance characteristics of the performance data; and a prediction module for predicting the degree of change of the performance data in a future time period based on the performance characteristics and using the regression prediction layer of the regression prediction model to issue an early warning to the target database.

[0012] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above-mentioned database performance prediction method.

[0013] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned database performance prediction method when executed by a processor.

[0014] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the steps of the above-mentioned database performance prediction method when executed by a processor.

[0015] In an embodiment of the present disclosure, in view of the dynamic, complex and nonlinear performance data of a target database, the present disclosure provides a database performance prediction method based on a data-driven perspective. The method takes into account the temporal relationship characteristics of the database performance data, and uses a deep learning network to fully mine the essential performance characteristics of the database performance data, thereby predicting future database performance changes. The method has the advantages of strong robustness, high prediction accuracy and strong generalization ability, and has certain practical significance and value for the daily operation and maintenance of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 A diagram schematically illustrates an application scenario of a database performance prediction method, apparatus, device, medium, and program product according to an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of a method for predicting database performance according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 3 A flowchart of a method for predicting the performance of a Gaussian database according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 4 A schematic diagram of a structure of a database performance prediction device according to an embodiment of the present disclosure is shown; and

[0021] Figure 5 A block diagram schematically shows an electronic device for a method for predicting database performance according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0024] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0025] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0026] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0027] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0028] Figure 1 The application scenario diagram of the database performance prediction method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0029] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0030] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0031] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0032] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0033] It should be noted that the database performance prediction method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the database performance prediction device provided in the embodiment of the present disclosure can generally be set in the server 105. The database performance prediction method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the database performance prediction device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0034] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0035] The following will be based on Figure 1 The scene described by Figure 2~Figure 3 The performance prediction method of a database in an embodiment of the present disclosure is described in detail.

[0036] First, the technical terms described in this article are explained and described as follows.

[0037] DBSCAN algorithm: Density-Based Spatial Clustering of Applications with Noise, DBSCAN, density-based clustering algorithm.

[0038] Deep Neural Networks: DNN, a deep neural network with more than n hidden layers.

[0039] Least Squares Support Vector Machine: Least Squares-Support Vector Regression, LSSVR, a function regression algorithm based on least squares support vector machine.

[0040] In an embodiment of the present disclosure, training a deep neural network includes: inputting the initial feature matrix into the deep neural network to obtain an output of the deep neural network; and when the difference between the output of the deep neural network and the true feature representation is greater than a first specified threshold, adjusting the parameters and structure of the deep neural network until the difference between the output of the deep neural network and the true feature representation is less than or equal to the first specified threshold.

[0041] In some exemplary embodiments, it is first necessary to obtain a training set, and the performance data in the training set can come from multiple channels. For example, performance data of historical time periods can be collected from other databases of the same type as the target database. These databases have similar architectural designs, load patterns, or application scenarios as the target database, so their historical performance data can provide valuable references for training deep neural networks. In addition to external data, performance data can also be collected from the non-testing stage of the target database (such as a production environment or a pre-production environment). These data are closer to actual application scenarios and can more accurately reflect the performance of the database under actual load.

[0042] After obtaining the training set, an initial feature matrix with time characteristics is constructed based on these performance data, and then the initial feature matrix containing the features to be learned is used as input and input into the pre-constructed deep neural network. Then, the deep neural network processes the input data based on its current parameter configuration (such as weights, biases, etc.) and structural design (such as the number of layers, the number of neurons in each layer, etc.), and outputs a predicted feature representation (performance feature). Then, by comparing the output of the deep neural network with the known true feature representation, the difference between the two is calculated. If the difference is greater than the preset first specified threshold, it means that the performance of the current deep neural network has not yet reached the preset standard and needs further adjustment. For example, according to optimization algorithms such as gradient descent, fine-tune parameters such as network weights and biases to reduce prediction errors. Or adjust the structure, that is, increase or decrease the number of network layers, change the number or type of neurons in each layer, and introduce new regularization methods. The adjusted deep neural network accepts the same or new initial feature matrix as input again, repeats the above prediction, evaluation and adjustment process, until a stable state is reached, that is, the difference between the output of the deep neural network and the true feature representation is less than or equal to the first specified threshold. At this point, the deep neural network has fully learned the feature representation of the data and completed the training process.

[0043] It should be noted that other deep learning networks can also replace the deep god-level network in the embodiment of the present disclosure, such as CNN (convolutional neural network), RNN (recurrent neural network), etc., but convolutional neural networks are mainly used in fields such as image recognition and face recognition, while recurrent neural networks are more suitable for processing sequence data. They are relatively basic neural networks in deep learning and have problems such as gradient disappearance. Therefore, when choosing a deep learning network architecture, comprehensive consideration can be made based on specific application scenarios, data characteristics, and task requirements.

[0044] It is understandable that deep neural networks gradually extract and transform the features of input data through a multi-layer structure, and can learn more complex and advanced feature representations during training. At the same time, by continuously adjusting the parameters and structure of the deep neural network, the network can learn more accurate data representation and feature mapping, thereby providing data support for subsequent performance prediction.

[0045] On the basis of the above embodiments, in this embodiment, based on the trained deep neural network, a regression prediction layer is integrated to construct the regression prediction model.

[0046] Optionally, the regression prediction layer can be a least squares support vector machine (LSSVR) regression layer. LSSVR is a regression method based on the principle of support vector machine. It finds the optimal regression function by solving a linear equation system, thereby avoiding the quadratic programming problem in traditional support vector machines. LSSVR has the advantages of high computational efficiency, high prediction accuracy and low model complexity, and is suitable for processing nonlinear regression problems.

[0047] For example, after the deep neural network training is completed, a least squares support vector machine regression layer can be integrated at its output end, so that the output of the deep neural network (i.e., the extracted feature representation) is passed as input to the least squares support vector machine regression layer to build a regression prediction model to achieve regression prediction from feature extraction. In the least squares support vector machine regression layer, a function regression algorithm is used to perform regression prediction on the input feature representation to obtain the predicted value of the target variable.

[0048] It can be understood that by combining the feature extraction capability of the deep neural network and the regression prediction capability of the regression prediction layer, the constructed regression prediction model has significantly improved the prediction accuracy when predicting the performance of the target database, and the complexity of the regression prediction model is low, and it can be applied and promoted in practical problems.

[0049] In an embodiment of the present disclosure, training the regression prediction model includes: inputting the performance characteristics output by the deep neural network into the regression prediction layer to obtain the degree of change of the performance data output by the regression prediction layer; using a loss function to calculate the difference between the statistical distribution of the predicted degree of change and the statistical distribution of the actual change; and when the difference between the statistical distribution of the degree of change and the statistical distribution of the actual change is greater than a second specified threshold, updating the parameters of the regression prediction model through an optimizer according to the gradient of the loss function until the difference is less than or equal to the second specified threshold.

[0050] Optionally, the loss function uses the root mean square error (RMSE) loss function to evaluate the performance of the model. The RMSE loss function calculates the mean of the square of the difference between the predicted value and the true value, and then takes the square root to obtain a dimensionless error value. The smaller the value, the higher the prediction accuracy of the model.

[0051] Exemplarily, the performance characteristics of the input data are extracted through a trained neural network, and then the performance characteristics are input into the regression prediction layer. After the regression prediction layer receives the performance characteristics, these characteristics are further processed and converted to predict the change trend (degree of change) of the performance data, such as the rate of change of the performance data over time, the magnitude of increase or decrease, etc. In order to evaluate the accuracy of the prediction, the difference between the statistical distribution of the predicted degree of change and the statistical distribution of the actual change can be calculated by the root mean square error loss function. If the calculated difference value is greater than the second specified threshold, the parameters of the regression prediction model can be iteratively updated through an optimizer (such as gradient descent method, Adam, etc.) according to the gradient of the loss function to gradually reduce the loss value. When the difference value is less than or equal to the second specified threshold, the training ends.

[0052] It is understandable that by continuously optimizing the parameters of the regression prediction model, a prediction model with better fitting effect and higher prediction accuracy can be obtained, and ultimately an accurate prediction of the target database performance can be achieved, helping database administrators or system operation and maintenance personnel to understand in advance the performance bottlenecks or pressure points that the database may face, so as to take corresponding optimization measures or resource allocation strategies.

[0053] Figure 2 The flowchart of the method for predicting database performance according to an embodiment of the present disclosure is schematically shown.

[0054] like Figure 2 As shown, the database performance prediction method of this embodiment includes operations S210 to S240.

[0055] In operation S210, performance data of a target database in a historical period is obtained.

[0056] In an embodiment of the present disclosure, before obtaining the performance data of the target database historical time period, the user's consent or authorization is obtained. For example, before operation S210, a request to obtain the performance data of the target database historical time period is issued to the user. If the user agrees or authorizes that the performance data of the target database historical time period can be obtained, operation S210 is performed.

[0057] For example, the target database may be a Gaussian database, which is a high-performance, highly available, secure and reliable relational database system widely used in data storage, processing and analysis tasks. In order to optimize its performance and ensure the stable operation of the system, the performance data of the Gaussian database may be accurately predicted to timely discover and resolve potential performance issues.

[0058] After the target database is determined, operations S211 to S212 are performed.

[0059] In operation S211, a main database of the target database is identified.

[0060] In operation S212, the main libraries are classified according to the identification information of the main libraries, and the performance data of the main libraries corresponding to the classification are obtained, wherein the identification information is used to characterize the attributes of each main library.

[0061] For example, take a target application in a data center as an example. In this process, the main database of the Gaussian database, that is, the database that serves as the main data storage and transaction processing center, can be identified by analyzing the configuration information and connection status of the database or querying specific metadata. The main database is classified according to its identification information (such as the database name, IP address, physical or virtual server, business application, etc.), so as to obtain the performance data of all Gaussian databases corresponding to the target application.

[0062] When collecting performance data, including but not limited to CPU usage, memory usage, standby RTO time (Recovery Time Objective, which means the time required for the system or application to recover to an acceptable service level after a disaster), number of synchronous threads, online session rate, disk utilization, number of sessions and number of connections, these parameters can fully reflect the operating status and performance of the database, and provide strong data support for subsequent performance analysis and optimization.

[0063] It can be understood that by using the identification information of the main database for classification, the performance data of each main database can be obtained in a targeted manner, thereby providing a solid foundation for performance analysis, bottleneck identification and formulation of optimization strategies for the target database.

[0064] After the performance data is obtained, operations S213 to S215 are performed.

[0065] In operation S213, the performance data is integrated to obtain a performance data set, and different density thresholds are determined according to the performance data set.

[0066] Specifically, the previously collected performance data is integrated into a unified performance data set, which contains performance indicators in multiple dimensions, such as CPU usage, memory usage, disk I / O speed, etc.

[0067] In operation S214, the performance data set is detected based on different density thresholds to identify abnormal values ​​in the performance data set.

[0068] In operation S215, outliers in the performance data set are removed to obtain remaining performance data of the performance data set.

[0069] Exemplarily, the DBSCAN algorithm is used to clean the data. The DBSCAN algorithm is a density-based clustering algorithm that can find clusters of any shape in a noisy spatial database. In the cleaning process, first, the statistical and machine learning methods can be used to determine the density threshold according to the distribution characteristics of the performance data for cluster analysis in the subsequent DBSCAN algorithm. Then, according to the set density threshold, the DBSCAN algorithm is applied to detect the integrated performance data set, and the data points are divided into core points, boundary points, and noise points, and the values ​​(i.e., outliers) that are out of the cluster compared with normal data points in the performance data set are identified. Finally, the outliers are removed to obtain the remaining performance data in the cleaned performance data set.

[0070] It should be noted that in actual operation, because different density thresholds may lead to different clustering results and outlier identification situations, different density thresholds can be used to perform multiple tests on the performance data set as needed. Through multiple tests, the clustering effect and the number of outliers under different thresholds can be compared, so as to select the most appropriate threshold combination to ensure the accuracy and effectiveness of data cleaning.

[0071] It is understandable that by cleaning data and eliminating outliers, more accurate and reliable data can be obtained, thereby more truly reflecting the performance status of the target database and avoiding interference with subsequent performance analysis and optimization work.

[0072] Based on the above embodiment, in this embodiment, the missing values ​​of the performance data set are identified and filled with the missing values ​​using the mean filling method; and for the performance data set with the missing values ​​filled, the performance data set is normalized using the maximum and minimum normalization method.

[0073] Exemplarily, the performance data set is carefully examined to identify all missing values. After the missing values ​​are identified, the missing values ​​can be filled using the mean filling method. The mean filling method is a simple and effective data filling method that calculates the mean based on other non-missing values ​​in the data set and assigns the mean to the missing value. The performance data set is then normalized, for example, by the maximum and minimum normalization method, that is, each data point is subtracted from the minimum value in the data set and then divided by the difference between the maximum and minimum values ​​in the data set, thereby obtaining a normalized value between 0 and 1 (or other specified range).

[0074] It can be understood that by identifying and filling missing values ​​and performing normalization, a Gaussian performance data set without dirty data can be obtained. This data set not only eliminates the impact of missing values ​​and dimensions on data analysis and decision-making, but also retains useful information in the data set, providing strong support for subsequent database performance analysis and optimization work.

[0075] In operation S220, an initial feature matrix with time characteristics is constructed based on the performance data.

[0076] According to the characteristics of Gaussian database performance data, it is classified according to identification information (such as server IP, i.e. main library IP), and a feature matrix with time characteristics is constructed for the performance data of each set of libraries as the input of the recursive deep neural network.

[0077] For a certain target database, if a performance indicator m of the target database at time t is , then the Gaussian performance data of the library at n moments is:

[0078]

[0079] Then the m performance indicators and the characteristic matrix of the library at n moments are expressed as

[0080]

[0081] In operation S230, the initial feature matrix is ​​input into a deep neural network in a pre-trained regression prediction model to extract performance features of the performance data.

[0082] In operation S240, based on the performance characteristics, the regression prediction layer of the regression prediction model is used to predict the degree of change of the performance data in a future time period, so as to issue an early warning to the target database.

[0083] In the embodiment of the present disclosure, a corresponding operation portal is provided for the user to choose to agree or reject the automated decision result. That is, before the process of predicting the degree of change of the performance data in the future time period is performed, an instruction of the user to agree or reject the process / decision is obtained through the corresponding operation portal. If the user agrees to perform the process / decision, the process / decision of predicting the degree of change of the performance data in the future time period is performed, that is, step S240 is executed. If the user refuses to perform the process / decision, the expert decision process is entered.

[0084] Figure 3 The flowchart of the performance prediction method of the Gaussian database according to the embodiment of the present disclosure is schematically shown.

[0085] In some exemplary embodiments, Figure 3 As shown in the figure, firstly, the original performance data of the Gaussian database is obtained, and then the density-based data clustering DBSCAN algorithm is used to clean the dirty data of the Gaussian performance data to obtain valuable data. After data cleaning, an initial feature matrix is ​​constructed for each performance data as the input of the deep learning network module. Then, the deep neural network is used to train the historical Gaussian feature data to capture the time dependency of the sequence data and deeply extract the performance characteristics of the Gaussian performance data. Finally, the thread regression prediction module, that is, the least squares support vector machine regression algorithm, is used to construct a linear regression function, so as to predict the data performance trend in the future time based on the historical data.

[0086] For example, the trained regression prediction model is used to input various performance data to predict the trend changes of Gaussian performance data in the future. For example, the change trend of query response time, if the query response time gradually increases, it means that the database needs to optimize the query statement, add indexes or upgrade the hardware configuration. For example, the change trend of throughput, if the throughput drops significantly in a specific time period, it means that the database administrator needs to increase the number of servers, optimize the database architecture or adopt a load balancing strategy. For example, the change trend of CPU usage, if the CPU usage continues to increase, it means that the database performance may decline, and it is necessary to optimize the query algorithm, reduce unnecessary computing operations or upgrade the CPU hardware. For example, the change trend of memory occupancy, if the memory occupancy fluctuates periodically, it means that the database needs to optimize the memory management strategy, such as using caching technology, adjusting memory allocation parameters, etc. For example, the change trend of disk I / O, if the disk I / O read and write speed slows down, it means that the database administrator needs to optimize the disk storage structure, increase the number of disks or use faster storage devices (such as SSD).

[0087] It can be understood that the disclosed embodiment, based on the data-driven perspective, proposes a performance prediction method for a database based on deep learning, targeting the complex nonlinearity and time characteristics of the target database performance indicators. The method aims to utilize the scalability, portability and strong adaptability of deep learning algorithms to process performance data in complex databases and predict the changing trends of database performance indicators. The method has the advantages of strong robustness, high prediction accuracy and strong generalization ability. Through this means, to a certain extent, early warning signals can be provided to production and operation personnel. When the predicted performance data of a certain database deviates from the normal indicator threshold, the production and operation personnel can analyze and deal with emergencies in advance, effectively ensuring the stable operation of the target database performance and avoiding the occurrence of production accidents to a certain extent.

[0088] Figure 4 The structural block diagram of the performance prediction device of a database according to an embodiment of the present disclosure is schematically shown.

[0089] like Figure 4 As shown, the database performance prediction device 400 according to this embodiment includes an acquisition module 410 , a construction module 420 , an extraction module 430 and a prediction module 440 .

[0090] The acquisition module 410 is used to acquire the performance data of the target database in the historical time period. In one embodiment, the acquisition module 410 can be used to perform the operation S210 described above, which will not be described in detail here.

[0091] The construction module 420 is used to construct an initial feature matrix with time characteristics based on the performance data. In one embodiment, the construction module 420 can be used to perform the operation S220 described above, which will not be described in detail here.

[0092] The extraction module 430 is used to input the initial feature matrix into the deep neural network in the pre-trained regression prediction model to extract the performance features of the performance data. In one embodiment, the extraction module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0093] The prediction module 440 is used to predict the degree of change of the performance data in the future time period based on the performance characteristics and using the regression prediction layer of the regression prediction model to warn the target database. In one embodiment, the prediction module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0094] In an embodiment of the present disclosure, the acquisition module 410 is specifically used to: identify the main library of the target database; and classify each of the main libraries according to the identification information of the main library, and obtain the performance data of the main library corresponding to the classification, wherein the identification information is used to characterize the attributes of each of the main libraries.

[0095] In an embodiment of the present disclosure, the acquisition module 410 can also be used to: integrate the performance data to obtain a performance data set, and determine different density thresholds based on the performance data set; detect the performance data set based on the different density thresholds to identify outliers in the performance data set; and eliminate outliers in the performance data set to obtain the remaining performance data of the performance data set.

[0096] In an embodiment of the present disclosure, the acquisition module 410 can also be used to: identify missing values ​​of the performance data set and fill the missing values ​​using the mean filling method; and for the performance data set with the missing values ​​filled, normalize the performance data set using the maximum and minimum normalization method.

[0097] In an embodiment of the present disclosure, the extraction module 430 may also be used to: input the initial feature matrix into the deep neural network to obtain the output of the deep neural network; and when the difference between the output of the deep neural network and the true feature representation is greater than a first specified threshold, adjust the parameters and structure of the deep neural network until the difference between the output of the deep neural network and the true feature representation is less than or equal to the first specified threshold.

[0098] In an embodiment of the present disclosure, the prediction module 440 may also be used to: integrate a regression prediction layer based on the trained deep neural network to construct the regression prediction model.

[0099] In an embodiment of the present disclosure, the prediction module 440 can also be used to: input the performance characteristics output by the deep neural network into the regression prediction layer to obtain the degree of change of the performance data output by the regression prediction layer; use the loss function to calculate the difference between the statistical distribution of the predicted degree of change and the statistical distribution of the actual change; and when the difference between the statistical distribution of the degree of change and the statistical distribution of the actual change is greater than a second specified threshold, according to the gradient of the loss function, update the parameters of the regression prediction model through the optimizer until the difference is less than or equal to the second specified threshold.

[0100] According to an embodiment of the present disclosure, any multiple modules of the acquisition module 410, the construction module 420, the extraction module 430 and the prediction module 440 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 410, the construction module 420, the extraction module 430 and the prediction module 440 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 410, the construction module 420, the extraction module 430 and the prediction module 440 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0101] Figure 5 A block diagram schematically shows an electronic device for a method for predicting database performance according to an embodiment of the present disclosure.

[0102] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0103] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0104] According to an embodiment of the present invention, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that the computer program read therefrom is installed into the storage portion 508 as needed.

[0105] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0106] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0107] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the database performance prediction method provided by the embodiment of the present disclosure.

[0108] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0109] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0110] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0111] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0112] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0114] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A database performance prediction method, characterized in that: The method comprises: Obtain the performance data of the target database in the historical period; Based on the performance data, construct an initial feature matrix with time characteristics; Inputting the initial feature matrix into a deep neural network in a pre-trained regression prediction model to extract performance features of the performance data; and Based on the performance characteristics, the regression prediction layer of the regression prediction model is used to predict the degree of change of the performance data in a future time period to issue an early warning to the target database.

2. The method according to claim 1, characterized in that The acquisition of the performance data of the target database in the historical time period includes: identifying a master repository of the target database; and According to the identification information of the main library, each of the main libraries is classified, and the performance data of the main library corresponding to the classification is obtained, wherein the identification information is used to characterize the attribute of each of the main libraries.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Integrate the performance data to obtain a performance data set, and determine different density thresholds based on the performance data set; Based on different density thresholds, detecting the performance data set to identify abnormal values ​​in the performance data set; and Outliers in the performance data set are removed to obtain remaining performance data of the performance data set.

4. The method according to claim 3, characterized in that The method further comprises: Identifying missing values ​​of the performance data set and filling the missing values ​​using a mean filling method; and For the performance data set that fills the missing values, the performance data set is normalized by using the maximum and minimum normalization method.

5. The method according to claim 1, characterized in that Training the deep neural network includes: Inputting the initial feature matrix into the deep neural network to obtain an output of the deep neural network; and When the difference between the output of the deep neural network and the true feature representation is greater than a first specified threshold, the parameters and structure of the deep neural network are adjusted until the difference between the output of the deep neural network and the true feature representation is less than or equal to the first specified threshold.

6. The method according to claim 1 or 5, characterized in that: The method further comprises: Based on the trained deep neural network, a regression prediction layer is integrated to construct the regression prediction model.

7. The method according to claim 6, characterized in that Training the regression prediction model includes: Inputting the performance characteristics output by the deep neural network into the regression prediction layer to obtain the degree of change of the performance data output by the regression prediction layer; Calculating the difference between the predicted statistical distribution of the degree of change and the actual statistical distribution of the degree of change using a loss function; and When the difference between the statistical distribution of the degree of change and the statistical distribution of the actual change situation is greater than a second specified threshold, the parameters of the regression prediction model are updated by the optimizer according to the gradient of the loss function until the difference is less than or equal to the second specified threshold.

8. A performance prediction device for a database, characterized in that: The device comprises: An acquisition module is used to obtain the performance data of the target database in a historical period; A construction module, used for constructing an initial feature matrix with time characteristics based on the performance data; an extraction module, configured to input the initial feature matrix into a deep neural network in a pre-trained regression prediction model to extract performance features of the performance data; and A prediction module is used to predict the degree of change of the performance data in a future time period based on the performance characteristics and using the regression prediction layer of the regression prediction model to issue an early warning to the target database.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images