Database load prediction method and device, server and storage medium

Abstract

The invention provides a database load prediction method and device, a server and a storage medium. The method comprises the steps of obtaining request data received by a database in response to a database load prediction request initiated by a service terminal; according to the access time information of the request data, extracting a time sequence feature of the request data, and obtaining service peak marking information of the request data to extract a service peak feature of the request data; inputting the time sequence characteristics of the request data and the business peak characteristics into a pre-acquired load prediction model; and processing the time sequence characteristics of the request data and the business peak characteristics based on the load prediction model, and outputting load information of the database, the load information including the query quantity per second of the database. Through the method, the prediction precision of the database load is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a database load prediction method, device, server and storage medium. Background Art

[0002] With the continuous development of big data, database systems have become the core components for storing large-scale data in financial enterprises. With the explosive growth of data volume and the continuous changes in business needs, database systems are facing increasing load pressure. In order to ensure the efficient operation and reliability of database systems, predicting database load information has become an important research field.

[0003] The current database prediction scheme mainly focuses on the improvement of prediction algorithms, whose input is historical data and output is future trends. For some applications, their load is related to the external service time of the application. For example, the load on weekends and holidays is much lower than that on ordinary working days. At the same time, the prediction algorithm is difficult to predict business peaks. For example, in the scenario of commemorative coin issuance, the system cannot predict the issuance time of the commemorative coins, resulting in inaccurate database load prediction at the issuance time.

[0004] Therefore, there is an urgent need to propose a database load prediction solution that can effectively solve the problem of insufficient prediction accuracy caused by different data request times and different businesses. Summary of the invention

[0005] The present application provides a database load prediction method, device, server and storage medium, which can effectively improve the accuracy of database load prediction.

[0006] In a first aspect, the present application provides a database load prediction method, comprising:

[0007] In response to a database load prediction request initiated by a business terminal, obtaining request data received by the database;

[0008] Extracting time series features of the request data according to access time information of the request data, and acquiring business peak mark information of the request data to extract business peak features of the request data;

[0009] Inputting the time series characteristics of the request data and the business peak characteristics into a pre-acquired load prediction model, wherein the load prediction model is a model trained based on historical request data and used to predict database load information;

[0010] The time series characteristics of the request data and the business peak characteristics are processed based on the load prediction model, and the load information of the database is output, where the load information includes the query volume of the database per second.

[0011] In one implementation, the acquiring the service peak mark information of the request data includes:

[0012] Transmitting the request data to the service terminal;

[0013] In response to the service peak mark added by the service terminal to the request data, the service peak mark information of the request data is obtained; wherein the service peak mark is the service peak mark added by the service terminal to the request data according to the service information corresponding to the request data.

[0014] In one embodiment, the method further comprises:

[0015] Collecting historical request data received by the database and its corresponding historical load information, and extracting time series characteristics and business peak characteristics of the historical request data;

[0016] The time series characteristics and business peak characteristics of the historical request data are used as an input feature set, and the historical load information is used as an output set to train an initial model to obtain the load prediction model.

[0017] In one embodiment, the initial model includes a time series prediction module and a machine learning module; the time series features and business peak features of the historical request data are used as an input feature set, and the historical load information is used as an output set to train the initial model, including:

[0018] Inputting the time series features of the historical request data into a time series prediction module, training the time series prediction module, so that the time series module outputs processed predicted time series features;

[0019] The predicted time series features output by the time series prediction module and the business peak features of the historical request data are used as an input feature set, and the historical load information is used as an output set to train the machine learning module.

[0020] In one embodiment, the training of the initial model using the time series features and business peak features of the historical request data as an input feature set and the historical load information as an output set includes:

[0021] The time series characteristics and business peak characteristics of the historical request data are used as an input feature set, and the historical load information is used as an output set. The initial model is iteratively trained until a preset number of iterations is reached or the loss value meets a preset loss condition, thereby obtaining the load prediction model.

[0022] In one embodiment, the method further comprises:

[0023] Obtain a predefined database table structure, where the database table structure includes time series features and marked business peak features carried by each request data;

[0024] The step of taking the time series features and the business peak features of the historical request data as an input feature set includes:

[0025] The time series characteristics and business peak characteristics of the historical request data are determined as the input feature set in the form of the database table structure.

[0026] In one embodiment, the method further comprises:

[0027] The load information of the database is fed back to the service terminal, so that the service terminal adjusts the processing priority of each service based on the load information of the database.

[0028] According to a second aspect of the present application, a database load prediction device is provided, comprising:

[0029] A data acquisition module, which is configured to respond to a database load prediction request initiated by a business terminal and acquire request data received by the database;

[0030] a feature extraction module, configured to extract time series features of the request data according to access time information of the request data, and obtain business peak mark information of the request data to extract business peak features of the request data;

[0031] An input module, configured to input the time series characteristics of the request data and the business peak characteristics into a pre-acquired load prediction model, wherein the load prediction model is a model trained based on historical request data and used to predict database load information;

[0032] A prediction module is configured to process the time series characteristics of the request data and the business peak characteristics based on the load prediction model, and output the load information of the database, wherein the load information includes the query volume of the database per second.

[0033] According to a third aspect of the present application, there is provided a server, comprising: a processor, and a memory communicatively connected to the processor;

[0034] The memory stores computer-executable instructions;

[0035] The processor executes the computer-executable instructions stored in the memory to implement the database load prediction method provided in any one of the first aspects above.

[0036] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, the method for predicting a database load as provided in any one of the first aspects above is used.

[0037] The database load prediction method, device, server and storage medium provided in the present application extract the time series characteristics of the request data and the business peak characteristics corresponding to the business peak marking information based on human-computer interaction, and use the model to process the time series characteristics and the business peak characteristics. The predicted database load is more in line with the actual situation, which effectively improves the accuracy of the database load prediction, facilitates users to predict database resources, thereby reasonably allocating resources, ensuring the stable operation of the system, and bringing a better experience to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] Figure 1 A possible scenario diagram provided for an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a flow chart of a database load prediction method provided in an embodiment of the present application;

[0041] Figure 3 for Figure 2 Schematic diagram of the process of step S202;

[0042] Figure 4 A schematic diagram of a flow chart of another database load prediction method provided in an embodiment of the present application;

[0043] Figure 5 A flowchart of another database load prediction method provided in an embodiment of the present application;

[0044] Figure 6a One of the flow charts of a data load prediction method provided by an exemplary embodiment of the present application;

[0045] Figure 6b A second flow chart of a data load prediction method provided by an exemplary embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of a database load prediction device provided in an embodiment of the present application;

[0047] Figure 8A schematic diagram of the structure of a server provided in an embodiment of the present application.

[0048] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of financial data or user data and other information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. It should be noted that in the embodiments of this application, some existing solutions in the industry such as certain software, components, models, etc. may be mentioned, and they should be considered as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0051] For ease of understanding, let's first briefly explain the two terms database and prediction:

[0052] Database and prediction are two different concepts, but they can be used together to achieve more accurate prediction and analysis. Databases are used to store and manage large amounts of data, while prediction can predict trends and results in the future based on historical data and current information. In terms of database prediction, various statistical methods and machine learning algorithms can be used to analyze the data stored in the database and predict future trends and results. For example, by analyzing historical sales data, future sales trends can be predicted; by analyzing user behavior data, future user behavior and needs can be predicted.

[0053] In order to achieve more accurate predictions, it is necessary to ensure the quality and accuracy of the data in the database. In addition, it is also very important to choose appropriate prediction models and methods, which can be selected according to the specific situation. In short, databases and predictions can be used in combination to achieve more accurate predictions and analysis. By analyzing the data stored in the database, you can better understand future trends and results and make corresponding decisions and actions.

[0054] Combine the following Figure 1 The possible application scenarios provided by the embodiments of the present application are introduced. Figure 1 This is an application scenario diagram of the database load prediction method provided in this embodiment. Figure 1 As shown, the application scenario includes: a prediction server 110, an application (server) 120 (the number may be one or more), a database (server) 130, and a business terminal 140. Optionally, the business terminal 140 may be a terminal device for bank staff to handle various banking services. A user may initiate a database load prediction request through the business terminal, for example, by triggering a database load prediction instruction through a human-computer interaction interface to send a database load prediction request to the prediction server 110. The prediction server 110 obtains the request data of each application 120 currently received by the database and performs database load prediction. In some embodiments, the business terminal 140 may also send a business request to the application server 120, and the application server 120 returns the execution result of the business request. During the business processing, the application server 120 may send query statements or commands to the database 130 according to the business logic requirements, and the database 130 executes these queries or commands and returns the results to the application server.

[0055] Among them, the above-mentioned server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; the above-mentioned terminals can include but are not limited to computers, smart phones, tablets, e-book readers, Moving Picture experts group audio layer III (MP3) players, Moving Picture experts group audio layer IV (MP4) players, portable computers, car computers, wearable devices, desktop computers, set-top boxes, smart TVs, etc.

[0056] In the process of predicting database load in related technologies, the focus is mainly on improving the prediction algorithm, whose input is historical data and output is future trends. For some applications, the load is related to the external service time of the application. For example, the load on weekends and holidays is much lower than the load on daily working days. At the same time, it is difficult for the prediction algorithm to predict the arrival of business peaks. For example, in the scenario of commemorative coin issuance, the system cannot predict the time of commemorative coin issuance, which will make it difficult to accurately predict the database load. The technical staff of this application takes into account the time characteristics of the data and the fact that users can know the arrival of the transaction peak in advance in certain business scenarios. By extracting the time series characteristics of the request data and the business peak characteristics corresponding to the business peak marking information based on human-computer interaction, the model is used to process the time series characteristics and business peak characteristics, and the predicted database load is more in line with the actual situation, which can effectively improve the accuracy of database load prediction.

[0057] based on Figure 1 In the application scenario shown in the figure, the embodiment of the present application provides a method for predicting database load, and the execution subject can be Figure 1 The prediction server 110 in . Figure 2 The process of the database load prediction method provided in the embodiment of the present application Figure 1 .like Figure 2 As shown, the method may include the following steps S201-S204:

[0058] Step S201: In response to a database load prediction request initiated by a service terminal, obtain request data received by a database.

[0059] In this embodiment, the user can initiate a database load prediction request to the prediction server through the service terminal. For example, when the user needs to process a certain service and the service volume is large and requires a higher database load, the user can initiate a database load request to the prediction server through the service terminal. The prediction server responds to the database load request and obtains the request data received by the database. For example, the request data can be the request data of a specific target application received by the database, and can request a timestamp, a request quantity, a request type, and can also include the database data query volume per second (QPS) corresponding to the current request, etc. Among them, the target application can be the corresponding application that the service terminal needs to process the service, or other applications.

[0060] It can be understood that in response to the conditions or states on which the executed operations depend, when the dependent conditions or states are met, one or more operations executed can be in real time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0061] Step S202: extracting time series features of the request data according to access time information of the request data, and acquiring business peak mark information of the request data to extract business peak features of the request data.

[0062] Among them, the access time information is the time information of accessing the database, which includes timestamps, such as year, month, week, day (whether it is a working day), etc. For example, the request data timestamps are organized into a time series to clearly identify the time point corresponding to each request data, thereby identifying the number of requests and trends in each time period.

[0063] The business peak marking information may be information that is marked by users based on human-computer interaction. For example, users may mark specific events (such as promotional activities, release dates) as business peaks. By obtaining the business peak marking information corresponding to the request data, it is possible to determine whether the corresponding request data has business peak features, and extract the corresponding business peak features, which may include the amount of requests required during the business peak period, duration, and so on.

[0064] Step S203: input the time series characteristics of the request data and the business peak characteristics into a pre-acquired load prediction model, where the load prediction model is a model trained based on historical request data and used to predict database load information.

[0065] Exemplarily, time series features and business peak features can be input into the model as a set of vectors (after normalization) for prediction, or they can be input into the model as respective feature vectors for training, which is not particularly limited in this embodiment.

[0066] It should be noted that the load prediction model can be trained in the prediction server using historical request data and corresponding database load information, or it can be obtained by the prediction server from other servers or devices specifically used for training models. The load prediction model can be a combined model based on a machine learning model and a time series prediction model, or it can be a separate model based on a machine learning model.

[0067] Step S204: Process the time series characteristics of the request data and the business peak characteristics based on the load prediction model, and output the load information of the database, wherein the load information includes the query volume per second of the database.

[0068] In the related art, the load prediction model usually predicts the database load information only based on the data request volume, while ignoring the time characteristics of the data request and the business peak characteristics when the bank processes the business, resulting in inaccurate prediction results of the model. Before the model prediction, this embodiment extracts the time series characteristics and business peak characteristics of the request data and inputs them into the model for prediction, which can effectively solve the prediction inaccuracy problem caused by the above scenarios.

[0069] It can be understood that the load information in this embodiment can be within a period of time in the future (technical personnel in this field can determine the time period in combination with actual applications or prior data. Similarly, the output set used in the model training process also corresponds to the load information within this time period). In some embodiments, the load information can also include the CPU and memory information of the database.

[0070] Next, the business peak mark information of the request data obtained in the above step S202 is further introduced. Figure 3 As shown, the following steps may be included: S2021, transmitting the request data to the business terminal; S2022, in response to the business peak mark added by the business terminal to the request data, obtaining the business peak mark information of the request data; wherein, the business peak mark is the business peak mark added by the business terminal to the request data according to the business information corresponding to the request data.

[0071] In this embodiment, it is considered that it is difficult for the prediction server to directly predict whether there is a business peak for the request data, and the user can know the arrival of the transaction peak in advance in the corresponding business scenario. For example, in the commemorative coin issuance scenario, the user can know the issuance time of the commemorative coin through other information channels or experience, thereby determining the business peak (the business peak is a period of time after the commemorative coin is issued, and this period of time can be determined based on prior data). By performing human-computer interaction on the request data in these business scenarios, the business peak can be marked, which is convenient for the prediction model to predict the database load at the business peak. In some embodiments, the business peak mark can also be directly marked by the user when the target application initiates the request data.

[0072] The above technical solution provided in the embodiment of the present application marks the business peak of the request data through human-computer interaction, which makes up for the defect that the algorithm cannot directly predict the business peak, thereby improving the accuracy of database load prediction.

[0073] Figure 4It is a flow chart of another database load prediction method provided by an embodiment of the present application. On the basis of the above embodiment, this embodiment trains a load prediction model on the prediction server side to realize rapid calling of the model. Specifically, in addition to the above steps S201-S203, the method of this embodiment can also include steps S401 and S402.

[0074] Step S401: collect historical request data received by the database and its corresponding historical load information, and extract time series characteristics and business peak characteristics of the historical request data.

[0075] In this embodiment, the historical request data may also be the historical request data initiated for the target application, so as to realize the prediction for the same application request, which can effectively improve the prediction accuracy of the model. The extraction process of the time series features and business peak features of the historical request data is similar to the extraction process of the time series features and business peak features in the above model processing process, and the relevant description will not be repeated here.

[0076] Step S402: Use the time series characteristics and business peak characteristics of the historical request data as an input feature set and the historical load information as an output set to train an initial model to obtain the load prediction model.

[0077] In this embodiment, the extracted time series features and business peak features are used as the input feature set of the model. These features help the model understand the time series pattern and peak behavior of the request data, and the historical load information is used as the output set of the model, which is the target that the model needs to predict, that is, the load of the system (in the future) under given request features. The input feature set and the output set are used to train an initial model. The purpose of this model is to learn the patterns in the historical data so that the system load under future requests can be predicted. Through the training process, the parameters of the model are adjusted so that it can accurately predict the load under different request features, thereby obtaining a more accurate load prediction model.

[0078] In some embodiments, in order to further improve the accuracy of the model, the initial model may include a time series prediction module and a machine learning module. The above step S402 uses the time series features of the historical request data and the business peak features as the input feature set and the historical load information as the output set to train the initial model, which may be done in the following manner:

[0079] Inputting the time series features of the historical request data into a time series prediction module, training the time series prediction module, so that the time series module outputs processed predicted time series features;

[0080] The predicted time series features output by the time series prediction module and the business peak features of the historical request data are used as an input feature set, and the historical load information is used as an output set to train the machine learning module.

[0081] In this embodiment, the time series prediction module can use time series analysis methods such as ARIMA algorithm or SARIMA algorithm to model and predict time series features. The machine learning module can use machine learning algorithms (such as random forest, support vector machine, neural network, etc.) to combine time series features and business peak features for training, and use the extracted features and features output by the time series module as input, and the load of historical request data (such as query volume per second) as output for supervised learning.

[0082] In the above-mentioned technical solution of the embodiment of the present application, a comprehensive load forecasting model is formed by combining the time series prediction module and the machine learning module. The model can better capture the impact of time series patterns and business peak characteristics on system load, thereby effectively improving the prediction performance of the model.

[0083] In some embodiments, the initial model is iteratively trained to achieve model convergence and improve model accuracy. Specifically, the above steps use the time series characteristics of the historical request data and the business peak characteristics as the input feature set, and use the historical load information as the output set to train the initial model, which can be done in the following way:

[0084] The time series characteristics and business peak characteristics of the historical request data are used as an input feature set, and the historical load information is used as an output set. The initial model is iteratively trained until a preset number of iterations is reached or the loss value meets a preset loss condition, thereby obtaining the load prediction model.

[0085] Exemplarily, the time series features and business peak features of historical request data are combined into an input feature set. The time series features can clarify the time-related patterns such as periodicity and trend of the request data, and the business peak features clarify the peak events or conditions (such as time points) of the request data in a marked form, and use the historical load information as the output set, that is, the actual load situation under the corresponding request features. By selecting an initial model, such as a machine learning model, a neural network model, or a comprehensive model including a time series prediction module and a machine learning module as in the above embodiment as the initial model. Then, the user can determine the number of iterations according to the actual application or prior data, enter the iterative training step, and in each iteration, input the input feature set into the model, the model outputs the predicted load information, compares the predicted load information output by the model with the actual historical load information, and calculates the loss value, wherein the loss value can be measured by the loss function (such as mean square error, cross entropy, etc.) to measure the accuracy of the prediction, and according to the loss value, adjust the model parameters by an optimization algorithm (such as gradient descent, Adam, etc.) to reduce the loss value and improve the prediction accuracy. When the loss value drops below a preset loss threshold (determined by technicians in this field based on prior data), the model is considered to have achieved satisfactory prediction accuracy, or when the number of iterations reaches the maximum number of iterations, the iteration process is terminated to prevent the training time from being too long. After the above iterative training, the model parameters are optimized, and a load prediction model that can accurately predict the load situation can be obtained.

[0086] In some embodiments, the input feature set may be further divided into a training set and a validation set, the training set may be used for training, and the validation set may be used for cross-validation of the model to select the optimal model.

[0087] The above-mentioned technical solution of the embodiment of the present application can gradually learn and optimize the model through iterative training, thereby improving the prediction ability of future loads and ensuring that the model has good performance and reliability in practical applications.

[0088] Figure 5 It is a flow chart of another database load prediction method provided by an embodiment of the present application. On the basis of the above embodiment, this embodiment uses a database table structure to construct an input feature set to facilitate model training and prediction, which can effectively improve the model processing performance. Specifically, in addition to the above steps S201-S203 and step S401, the method provided in this embodiment can also include step S501, and step S402 is further divided into step S4021.

[0089] Step S501: Obtain a predefined database table structure, where the database table structure includes time series features and marked business peak features carried by each request data.

[0090] In this embodiment, the database table structure may include time series features such as timestamp, hour, day of the week, whether it is a weekend, etc., and business peak features, such as whether it is a special event period (such as a release date), etc. These features are used to mark business peaks that may affect the request volume. Exemplarily, the time series features and business peak features may be shown in Table 1 below:

[0091] Table 1

[0092] Application Name Week Start time End Time Peak hour begins Peak hour ends Time Zone Order Processing on Monday 8:00 18:00 12:00 14:00 / Order Processing Tuesday 8:00 18:00 12:00 14:00 / Order Processing Wednesday 8:00 18:00 12:00 14:00 / Order Processing Thursday 8:00 18:00 12:00 14:00 / Order Processing Friday 8:00 18:00 12:00 14:00 / Order Processing Saturday 10:00 16:00 / / / Order Processing Sunday / / / / /

[0093] By pre-defining the table structure, we can ensure that all data are stored in a consistent format and standard to facilitate subsequent data processing and analysis.

[0094] Step S4021: determine the time series characteristics and business peak characteristics of the historical request data in the form of the database table structure as the input feature set, and use the historical load information as the output set to train an initial model to obtain the load prediction model.

[0095] In some embodiments, the database table structure may also carry the number of queries per second (QPS) corresponding to each request data. For example, the load statistics based on the business service time are shown in Table 2 below:

[0096]

[0097]

[0098] For example, QPS data is stored in a database table, and the table structure can be as follows: CREATE TABLE qps_data (timestamp TIMESTAMP PRIMARY KEY, qps INTEGER); the table contains two initial fields, timestamp records the timestamp of data insertion, and serves as the primary key. QPS records the number of requests per second corresponding to the timestamp, and inserts QPS data once every minute, recording the current time and the current number of requests.

[0099] In the table structure, for timestamps, corresponding date and time features are added and stored, such as hour (0-23), day of the week (0 for Sunday, 6 for Saturday), whether the timestamp is a weekend, whether the timestamp is a holiday, whether the timestamp is within normal service hours (for example, 9:00 to 17:00), etc. In addition, manually marked business peak features are added, for example, a new column is_special_event is added to the data table using the ALTER TABLE statement to mark request data during special events, which can be determined by comparing whether the date part of the timestamp is in a predefined special event date list.

[0100] Using the above database table structure, by storing various feature data in the form of database tables, including time features and manually marked business peak features, the data is organized and labeled more systematically, and the features are clearer, which facilitates more complex data analysis and model training to identify patterns, predict loads, and optimize system performance.

[0101] In some embodiments, by feeding back the predicted load information to the service terminal, the service terminal can effectively optimize resource allocation and improve system efficiency by adjusting the processing priority of the service. Specifically, the method provided in this embodiment can also include the following steps:

[0102] The load information of the database is fed back to the service terminal, so that the service terminal adjusts the processing priority of each service based on the load information of the database.

[0103] Exemplarily, the database load information may include the number of queries per second (QPS), remaining CPU, and remaining memory of the database in the future. Based on the load information, the service terminal may apply a predefined strategy to adjust the processing priority of each service. For example, when the database load is high (technical personnel in this field can determine based on prior data that when the QPS reaches a certain amount, the database load is considered to be high, and the same is true for other indicators), the priority of non-critical tasks can be reduced, and certain requests can be postponed or limited to ensure the smooth execution of critical tasks. Non-critical tasks and critical tasks can be set adaptively by the user, and when the load is low (similar to the determination process of high load), the priority of more tasks can be increased to make full use of system resources and improve the overall throughput.

[0104] To facilitate further understanding of the embodiments of the present application, the prediction process based on the business service period is as follows: Figure 6aAs shown, in this prediction process, in order to improve the prediction accuracy, statistics can be collected for a period of time (request data, i.e., historical data in the figure) before starting the prediction, and the prediction can be made in combination with the service time of each request data. Specifically, the designed human-computer interaction medium (data structure) is used, including the system's business service period, weekdays / weekends, holidays and other factors, and the business activity time that may bring about the transaction peak, etc. On this basis, the algorithm model is used to carry out prediction to improve the accuracy of the prediction.

[0105] Furthermore, in order to more accurately predict the load information such as the number of queries per second (QPS) of the database, and taking into account the special business peaks marked by weekends, holidays, external service hours, and users through terminal interaction, a method combining time series analysis with external factor marking can be used. Figure 6b As shown, the following process may be included:

[0106] S1. Data collection and preprocessing: Collect historical request data, which may include QPS data per minute, and mark the corresponding timestamps, and can mark the dates of special activities and specific time periods. S2. Feature engineering: Convert timestamps into date and time features, such as hours, weeks, whether it is a weekend, whether it is a public holiday, whether it is an external service period, etc., and add business peak features manually marked by the administrator. S3. Prediction model selection: Use time series prediction models combined with machine learning models for prediction, combine time series analysis and machine learning models, select a combination of one or more models for prediction, and assign weights to each model. S4. Model training and validation: Train the model and perform cross-validation to select the optimal model. S5. Prediction and result analysis: Use the trained model to predict QPS and output the prediction results.

[0107] The above technical solution of the embodiment of the present application is based on the database load prediction process of human-computer interaction, and the data structure for recording database load, system load and service time. The feature engineering takes into account weekends, holidays, external service periods and manually marked special business peak characteristics, and combines time series analysis and machine learning models to effectively predict the QPS of the database. The above solution can record the database's QPS, CPU, memory and other load information according to each application and each database, and on this basis, combined with the characteristics of each application, the business access time and business peak of each application, the knowledge and experience of the system administrator and the understanding of the operation and maintenance system for which he is responsible, the algorithm can help the administrator to better understand the future QPS trend, the system load trend, and whether the resources can meet the possible future business peaks. Predict, reasonably allocate resources, and ensure the stable operation of the system.

[0108] Figure 7 is a schematic diagram of the structure of a database load prediction device provided in an embodiment of the present application, such as Figure 7 As shown, the device 700 may include a data acquisition module 701, a feature extraction module 702, an input module 703 and a prediction module 704, wherein:

[0109] A data acquisition module 701 is configured to respond to a database load prediction request initiated by a service terminal and acquire request data received by the database;

[0110] A feature extraction module 702 is configured to extract time series features of the request data according to access time information of the request data, and obtain business peak mark information of the request data to extract business peak features of the request data;

[0111] An input module 703, which is configured to input the time series characteristics of the request data and the business peak characteristics into a pre-acquired load prediction model, where the load prediction model is a model trained based on historical request data and used to predict database load information;

[0112] The prediction module 704 is configured to process the time series characteristics of the request data and the business peak characteristics based on the load prediction model, and output the load information of the database, wherein the load information includes the query volume per second of the database.

[0113] In one implementation, the feature extraction module 702 includes:

[0114] a transmission unit, configured to transmit the request data to the service terminal;

[0115] A tag acquisition unit is configured to acquire the business peak tag information of the request data in response to the business peak tag added by the business terminal to the request data; wherein the business peak tag is the business peak tag added by the business terminal to the request data according to the business information corresponding to the request data.

[0116] In one embodiment, the device further comprises:

[0117] A collection module, which is configured to collect historical request data received by the database and its corresponding historical load information, and extract time series characteristics and business peak characteristics of the historical request data;

[0118] The training module is configured to use the time series characteristics and business peak characteristics of the historical request data as an input feature set and the historical load information as an output set to train an initial model to obtain the load prediction model.

[0119] In one embodiment, the initial model includes a time series prediction module and a machine learning module; the training module includes:

[0120] A first training unit is configured to input the time series features of the historical request data into a time series prediction module, and train the time series prediction module so that the time series module outputs processed predicted time series features;

[0121] The second training unit is configured to use the predicted time series features output by the time series prediction module and the business peak features of the historical request data as an input feature set, and use the historical load information as an output set to train the machine learning module.

[0122] In one embodiment, the training module includes:

[0123] The third training unit is configured to use the time series characteristics and business peak characteristics of the historical request data as an input feature set, and the historical load information as an output set, to iteratively train the initial model until a preset number of iterations is reached or the loss value meets a preset loss condition, thereby obtaining the load prediction model.

[0124] In one embodiment, the device further comprises:

[0125] A table structure acquisition module, which is configured to acquire a predefined database table structure, wherein the database table structure includes a time series feature and a marked business peak feature carried by each request data;

[0126] The taking the time series characteristics and business peak characteristics of the historical request data as the input feature set specifically includes: determining the time series characteristics and business peak characteristics of the historical request data as the input feature set in the form of the database table structure.

[0127] In one embodiment, the device further comprises:

[0128] A feedback adjustment module is configured to feed back the load information of the database to the service terminal, so that the service terminal adjusts the processing priority of each service based on the load information of the database.

[0129] The above-mentioned device provided in the embodiment of the present application can be used to execute the technical solution of the database load prediction method in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0130] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the feature extraction module 702 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above feature extraction module 702. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0131] Figure 8 is a schematic diagram of the structure of a server provided in an embodiment of the present application, such as Figure 8 As shown, the server may include: a transceiver 801 , a processor 802 , and a memory 803 .

[0132] The processor 802 executes the computer execution instructions stored in the memory, so that the processor 802 executes the scheme in the above embodiment. The processor 802 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0133] The memory 803 is connected to the processor 802 via a system bus and implements communication therebetween. The memory 803 is used to store computer program instructions.

[0134] The transceiver 801 may be used to perform data transmission with an external device.

[0135] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0136] The above-mentioned server provided in the embodiment of the present application can be used to execute the technical solution of the database load prediction method in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0137] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the database load prediction method corresponding to the above method embodiment.

[0138] The above-mentioned medium provided in the embodiment of the present application can be used to execute the technical solution of the database load prediction method in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0139] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the database load prediction method in the above embodiment.

[0140] The above-mentioned medium provided in the embodiment of the present application can be used to execute the technical solution of the database load prediction method in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0141] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the database load prediction method of the above embodiment.

[0142] The above-mentioned medium provided in the embodiment of the present application can be used to execute the technical solution of the database load prediction method in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0143] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the database load prediction method in the above embodiment can be implemented.

[0144] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0145] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0146] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0147] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0148] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0149] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0150] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0151] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0152] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0153] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A database load prediction method, characterized in that: include: In response to a database load prediction request initiated by a business terminal, obtaining request data received by the database; Extracting time series features of the request data according to access time information of the request data, and acquiring business peak mark information of the request data to extract business peak features of the request data; Inputting the time series characteristics of the request data and the business peak characteristics into a pre-acquired load prediction model, wherein the load prediction model is a model trained based on historical request data and used to predict database load information; The time series characteristics of the request data and the business peak characteristics are processed based on the load prediction model, and the load information of the database is output, where the load information includes the query volume of the database per second.

2. The method according to claim 1, characterized in that The obtaining of the business peak mark information of the requested data includes: Transmitting the request data to the service terminal; In response to the service peak mark added by the service terminal to the request data, the service peak mark information of the request data is obtained; wherein the service peak mark is the service peak mark added by the service terminal to the request data according to the service information corresponding to the request data.

3. The method according to claim 1 or 2, characterized in that: Also includes: Collecting historical request data received by the database and its corresponding historical load information, and extracting time series characteristics and business peak characteristics of the historical request data; The time series characteristics and business peak characteristics of the historical request data are used as an input feature set, and the historical load information is used as an output set to train an initial model to obtain the load prediction model.

4. The method according to claim 3, characterized in that The initial model includes a time series prediction module and a machine learning module; the time series features and business peak features of the historical request data are used as an input feature set, and the historical load information is used as an output set to train the initial model, including: Inputting the time series features of the historical request data into a time series prediction module, training the time series prediction module, so that the time series module outputs processed predicted time series features; The predicted time series features output by the time series prediction module and the business peak features of the historical request data are used as an input feature set, and the historical load information is used as an output set to train the machine learning module.

5. The method according to claim 3, characterized in that: The method of using the time series features and business peak features of the historical request data as an input feature set and the historical load information as an output set to train an initial model includes: The time series characteristics and business peak characteristics of the historical request data are used as an input feature set, and the historical load information is used as an output set. The initial model is iteratively trained until a preset number of iterations is reached or the loss value meets a preset loss condition, thereby obtaining the load prediction model.

6. The method according to claim 4 or 5, characterized in that: Also includes: Obtain a predefined database table structure, where the database table structure includes time series features and marked business peak features carried by each request data; The step of taking the time series features and the business peak features of the historical request data as an input feature set includes: The time series characteristics and business peak characteristics of the historical request data are determined as the input feature set in the form of the database table structure.

7. The method according to claim 1, characterized in that Also includes: The load information of the database is fed back to the service terminal, so that the service terminal adjusts the processing priority of each service based on the load information of the database.

8. A database load prediction device, characterized in that: include: A data acquisition module, which is configured to respond to a database load prediction request initiated by a business terminal and acquire request data received by the database; a feature extraction module, configured to extract time series features of the request data according to access time information of the request data, and obtain business peak mark information of the request data to extract business peak features of the request data; An input module, configured to input the time series characteristics of the request data and the business peak characteristics into a pre-acquired load prediction model, wherein the load prediction model is a model trained based on historical request data and used to predict database load information; A prediction module is configured to process the time series characteristics of the request data and the business peak characteristics based on the load prediction model, and output the load information of the database, wherein the load information includes the query volume of the database per second.

9. A server, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the database load prediction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the database load prediction method according to any one of claims 1 to 7.

Images