Database scaling processing method and device, computer equipment and readable medium

Abstract

The application provides a database scale processing method and device, computer equipment and readable medium. The method comprises the following steps: obtaining historical load data, current load data and current scale information of the database; predicting the load data of the database at a future time t according to the current load data, the historical load data and a pre-trained load prediction model; obtaining the scale quantity of the database at the future time t according to the current scale information and the load data at the future time t; and performing scale processing on the database according to the scale quantity. The technical scheme of the application can automatically perform scale processing on the database without human intervention throughout the process, thereby effectively saving labor costs, saving time and effort throughout the process, responding very quickly, effectively improving the efficiency of scale processing, and being very accurate without human prediction.

Description

[0001] The present application relates to the technical field of computer application, and in particular to a database capacity expansion and contraction processing method and device, computer equipment and readable medium.

[0002] Most of the databases in the industry currently support capacity expansion and contraction to adapt to changes in business traffic. When business grows rapidly, the existing database size may not be able to meet the business growth needs, or for some database traffic surge business scenarios, such as promotional activities, the database can be expanded to improve performance at this time. When the business shrinks or after the peak of the activity traffic, most of the resources of the existing database are in an idle state, and the database should be contracted to recover idle resources and save costs.

[0003] In the prior art, the capacity expansion and contraction of the database is mostly completed in advance by a database administrator (DBA). For example, the database traffic regularly (weekly or monthly) has a traffic peak, and the traffic value is several or tens of times higher than usual. If the DBA regularly expands the capacity before the traffic peak and contracts the capacity after the peak, the capacity of the database can be manually expanded and contracted based on the changes in business traffic.

[0004] However, the capacity expansion and contraction of the database in the prior art is based on the rough prediction of the DBA, and due to the involvement of human labor, the capacity expansion and contraction process is time-consuming and labor-intensive, and prediction errors may occur. Therefore, the efficiency and accuracy of the existing capacity expansion and contraction process are low.

[0005] The present application provides a database capacity expansion and contraction processing method and device, computer equipment and readable medium, which can improve the efficiency and accuracy of capacity expansion and contraction processing.

[0006] The present application provides a database capacity expansion and contraction processing method, which comprises:

[0007] Obtain historical load data, current load data and current scale information of the database;

[0008] According to the current load data, the historical load data and the pre-trained load prediction model, predict the load data of the database at a future time t;

[0009] According to the current scale information and the load data at the future time t, obtain the capacity expansion and contraction number of the database at the future time t;

[0010] ​​​According to the number of expansion and contraction, the database is processed by expansion and contraction.

[0011] The application provides a database expansion and contraction processing device, which comprises:

[0012] A data acquisition module is configured to acquire historical load data, current load data and current scale information of the database.

[0013] A prediction module is configured to predict load data of the database at a future time t according to the current load data, the historical load data and a pre-trained load prediction model.

[0014] An expansion and contraction number acquisition module is configured to acquire an expansion and contraction number of the database at the future time t according to the current scale information and the load data of the database at the future time t.

[0015] An expansion and contraction processing module is configured to process the database by expansion and contraction according to the expansion and contraction number.

[0016] The application further provides a computer device, which comprises:

[0017] One or more processors;

[0018] A memory configured to store one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the database expansion and contraction processing method as described above.

[0020] The application further provides a computer readable medium having a computer program stored thereon, which is executed by a processor to implement the database expansion and contraction processing method as described above.

[0021] The database expansion and contraction processing method and device, the computer device and the readable medium of the application can automatically process the database by expansion and contraction, without human intervention, thereby effectively saving labor cost, saving time and effort and improving the efficiency of expansion and contraction processing.

DRAWINGS

[0022] Figure 1 The flowchart of the database expansion and contraction processing method of the application.

[0023] Figure 2 The flowchart of the database expansion and contraction processing method of the application.

[0024] Figure 3 This is a structural diagram of a first embodiment of the database scaling processing device of the present invention.

[0025] Figure 4 This is a structural diagram of a second embodiment of the database scaling processing device of the present invention.

[0026] Figure 5 This is a structural diagram of an embodiment of the computer device of the present invention.

[0027] Figure 6 An example diagram of a computer device provided by the present invention.

Detailed Implementation Methods

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] Figure 1 This is a flowchart of an embodiment of the database scaling method of the present invention. Figure 1 As shown, the database scaling method in this embodiment may specifically include the following steps:

[0030] S100: Obtain historical load data, current load data, and current scale information from the database;

[0031] The database scaling method in this embodiment is implemented by a database scaling device, which is connected to the database and is used to automatically scale the database.

[0032] Specifically, in this embodiment, it is necessary to continuously collect and store the database load data in real time, which can be done at fixed time intervals. The collection interval can vary depending on the database type. For example, for Online Transaction Processing (OLTP) databases, which have high concurrency and fast performance index changes, the interval can be set to 10 seconds; for Online Analytical Processing (OLAP) databases, which have relatively slow performance index changes, the interval can be set to 1 minute.

[0033] The load data collected in the embodiment can include the operating system workload data of the machine where the database is located and the database workload data, such as the operating system workload data can include CPU consumption, disk I / O consumption, disk capacity, memory consumption, etc.; the database workload data can include the database throughput, etc. Since the current database is generally large in size, in order to more efficiently collect data, the embodiment further includes a data collection module. Secondly, in the embodiment, in order to predict the load data at future time t, not only the current load data of the database needs to be obtained, but also the historical load data needs to be combined, so the load data collected by the data collection module needs to be stored, therefore, the embodiment can further include a data storage module.

[0034] The data collection module can be subdivided into Agent end and Server end. The Agent end is deployed on the host machine of the database, responsible for the real data collection work, and pushes the data to the Server end. The Server end is responsible for receiving the data pushed by the Agent end, and processing, summarizing and storing it to the data storage module. The data storage module aims to store the workload data collected by the data collection module, wherein the load data at each time stored in the format of {timestamp, load data} so that these data can be read at any time. That is, the data collected by the data collection module is time-sequenced, and a time series database can be used as the data storage module to store the load data collected by the data collection module, which can efficiently store and quickly process massive time-sequenced big data, and can improve the efficiency of the module.

[0035] For example, when performing the expansion and contraction processing of the database, the current load data and the historical load data of the database at the current time can be obtained from the data storage module according to step S100. The historical load data is the load data of a historical time period before the current time according to actual needs. For example, the current load data can be represented as {current timestamp, current load data}; the historical load data can include {timestamp 1, load data 1}, {timestamp 2, load data 2}, …, {timestamp N, load data N} in order from front to back according to the timestamp. For different databases, the length of the historical time period included in the historical load data can not be the same.

[0036] In the embodiment, the current size information of the database also needs to be obtained, which specifically refers to the current capacity of the database.

[0037] In addition, it should be noted that the database in the embodiment can be a stand-alone database or a distributed database. If it is a distributed database, the sum of the loads of all machines of the distributed database can be taken as the load data of the database in the embodiment. For example, a certain distributed database is deployed on four machines, and at a certain moment, the CPU consumption of each machine is 60%, and the CPU consumption of the entire distributed database can be represented as 240% at this moment. Similarly, other load data of the distributed database can be represented in a similar manner. Alternatively, the average load of each machine can be taken as the load data of the database. In summary, the selection criteria of the load data at each moment should be consistent.

[0038] S101, predicting the load data of the database at a future moment t according to the current load data, the historical load data, and the pre-trained load prediction model;

[0039] This step is used to predict the load data at a future moment t. Specifically, the future moment t can be any moment after the current moment.

[0040] For example, the step S101 predicts the load data of the database at a future moment t according to the current load data, the historical load data, and the pre-trained load prediction model, and specifically can include:

[0041] (a1) predicting the first load data of the database at a future moment t according to the current load data, the historical load data, and a first load prediction model; the first load prediction model is a prediction model based on non-periodic change trend;

[0042] (b1) predicting the second load data of the database at the future moment t according to the current load data, the historical load data, and a second load prediction model; the second load prediction model is a prediction model based on periodic change trend; and / or

[0043] (c1) predicting the third load data of the database at the future moment t according to the current load data, the historical load data, and a third load prediction model; the third load prediction model is a prediction model based on holidays or emergencies;

[0044] Then, the first load data, the second load data, and / or the third load data of the database at the future moment t are accumulated as the load data of the database at the future moment t based on steps (a1)-(c1).

[0045] As described in the above steps, when predicting the load data of the database at a future moment t, at least one of the above steps (a1)-(c1) can be included, and if two or three are included, the sum of the prediction results of the included steps is taken as the load data of the database at the future moment t.

[0046] That is, the present embodiment can employ three models, i.e., the first load prediction model, the second load prediction model and the third load prediction model, to predict the load data at the future time t. The first load prediction model is a trend model, which does not consider the periodic change of the load data and the special rules of holidays. The second load prediction model is a model that is biased to predict periodic data, and can predict the load data at the future time t based on the historical periodic data. The third load prediction model is a model that is biased to predict data of holiday rules. In practical applications, preferably, the three prediction models are simultaneously employed to make up for each other, so as to increase the accuracy of the predicted load data at the future time t. At this time, the predicted load data Y at the future time t can be divided into three parts: a trend term f(x), a periodic term g(x) and a holiday term h(x), and the prediction result can be expressed as:

[0047] Y = f(x) + g(x) + h(x)

[0048] Among them, the trend term f(x) represents the change trend of the prediction result that is not periodic; the periodic term g(x) represents the change trend of the prediction result that is periodic, which can be a week, a month, a quarter or a year as a periodic unit; and the holiday term h(x) represents whether the day is a holiday or a historical sudden event occurs, etc. The prediction result of the present embodiment fits these terms, and then adds them up to obtain the final load data at the future time t.

[0049] In this embodiment, the historical load data of the database obtained in step S100 is relatively rich, for example, it can include historical load data of two months, one quarter, one year or other time length before the current timestamp. When predicting in step S101, the specific required historical load data selected by different models is not the same. For example, the first load prediction model is a trend item prediction model, at this time, the required historical load data can include load data of one week or one month before the current timestamp according to the requirement. The second load prediction model is a periodic item prediction model, at this time, the required historical load data can be obtained from the historical load data obtained in S100, which is the load data of the same time point in different time periods. For example, to predict the data of the xth hour after one day, the data of the xth hour of each day in history can be obtained from the historical event data. In this way, the accuracy of the prediction of data with periodic regularity can be improved. The third load prediction model is a holiday item prediction model, and the required historical load data is the load data of one, two or more holidays before the current timestamp according to the requirement, and this model can accurately predict the load data of the future holiday item. In practical application, the second load prediction model and the third load prediction model can predict the data of any time in the future, but the accuracy of the prediction of data meeting their own prediction rules is higher.

[0050] Further optionally, in this embodiment, the first load prediction model, the second load prediction model and the third load prediction model also need to be trained respectively.

[0051] Wherein when the first load prediction model is trained, a plurality of training data can be collected from the database storage module, each of the training data corresponds to a selected reference timestamp, including the current load data corresponding to the reference timestamp, the historical load data and the real load data of the future time t to be predicted after the reference timestamp. The historical load data includes a continuous historical time sequence of load data. In the specific training, the current load data and the historical load data in each training data are input into the first load prediction model, and the model can predict the predicted load data of the future time t after the reference timestamp. Then, whether the real load data and the predicted load data of the future time t after the reference timestamp are consistent is compared. If not, the parameters of the first load prediction model are adjusted based on the real load data and the predicted load data of the future time t after the reference timestamp. In the above manner, the first load prediction model is continuously trained by using a plurality of training data, until the real load data and the predicted load data of the future time t after the reference timestamp are always consistent in the continuous preset number of training, it can be considered that the first load prediction model has been trained well and can accurately predict the load data of the future time t, at this time the parameters of the first load prediction model can be determined, and the first load prediction model is determined.

[0052] Wherein when the second load prediction model is trained, a plurality of training data can be collected from the database storage module, each of the training data corresponds to a selected reference timestamp, including the current load data corresponding to the reference timestamp, the historical load data and the real load data of the same time point in the future period to be predicted after the reference timestamp. The historical load data, i.e. the load data of the same time point in different periods in history, includes a group of load data with periodic and discrete time points. In the specific training, the current load data and the load data of the same time point in different periods in history in each training data are input into the second load prediction model, and the model can predict the predicted load data of the same time point in the future period after the reference timestamp. Then, whether the real load data and the predicted load data of the same time point in the future period after the reference timestamp are consistent is compared. If not, the parameters of the second load prediction model are adjusted based on the real load data and the predicted load data of the same time point in the future period after the reference timestamp. In the above manner, the second load prediction model is continuously trained by using a plurality of training data, until the real load data and the predicted load data of the same time point in the future period after the reference timestamp are always consistent in the continuous preset number of training, it can be considered that the second load prediction model has been trained well and can accurately predict the load data of the same time point in the future period after the reference timestamp, at this time the parameters of the second load prediction model can be determined, and the second load prediction model is determined.

[0053] In the training of the third load prediction model, a plurality of training data can be collected from the database storage module, each of which corresponds to a selected reference timestamp and includes the current load data corresponding to the reference timestamp, the historical load data of a plurality of holidays, and the real load data of a specified time in a future holiday after the reference timestamp. The historical load data includes the time series load data of one, two or more holidays. In the specific training, the current load data and the historical load data of a plurality of holidays in each training data are input into the second load prediction model, which can predict the predicted load data of the specified time in the future holiday after the reference timestamp. Then, whether the real load data of the specified time in the future holiday after the reference timestamp is consistent with the predicted load data is compared. If not, the parameters of the third load prediction model are adjusted based on the real load data of the specified time in the future holiday after the reference timestamp and the predicted load data. In this way, the third load prediction model is continuously trained with a plurality of training data until the real load data of the specified time in the future holiday after the reference timestamp is consistent with the predicted load data in a plurality of preset times of training. It can be considered that the third load prediction model has been trained and can accurately predict the load data of the specified time in the future holiday after the reference timestamp. At this time, the parameters of the third load prediction model can be determined, and the third load prediction model can be determined.

[0054] In the embodiment, the more training data collected in the training of each model, the higher the accuracy of the trained model. For example, in practical applications, the number of training data can reach millions.

[0055] S102, according to the current scale information and the load data of the future time t, obtaining the expansion and contraction quantity of the database at the future time t;

[0056] The future time t in the embodiment can be any time after the current time. For example, in the embodiment, the implementation of the step can include two ways:

[0057] The first way: according to the current scale information and the load data of the future time t, obtaining the expansion and contraction quantity of the database at the future time t from the expansion and contraction configuration library;

[0058] Each piece of expansion and contraction configuration information in the expansion and contraction configuration library of the embodiment includes the corresponding relationship among the load data, the scale information before expansion and contraction, and the expansion and contraction quantity.

[0059] The implementation of the mode is that the expansion and contraction configuration information similar to the given workload is directly searched from the expansion and contraction configuration library, and if the matched expansion and contraction configuration information is found, the expansion and contraction is performed according to the expansion and contraction quantity indicated by the expansion and contraction configuration information. For example, the expansion and contraction configuration information format in the embodiment can be (CPU consumption, disk I / O consumption, disk capacity, memory consumption, database throughput, existing database size, and expansion and contraction quantity). If the expansion and contraction quantity is positive, it indicates the quantity of expansion; if the expansion and contraction quantity is negative, it indicates the quantity of contraction.

[0060] The expansion and contraction configuration information in the expansion and contraction configuration library is configured according to the actual working scene, is tested by the actual working scene, and can meet the working demand.

[0061] The second mode is that the expansion and contraction quantity of the database at the future time t is predicted according to the current size information, the load data at the future time t, and the pre-trained expansion and contraction configuration model.

[0062] Compared with the first mode, the mode is more intelligent. In the first mode, there can be no matched expansion and contraction configuration information. In the implementation mode, for any load data, the corresponding expansion and contraction quantity can be obtained.

[0063] S103, expansion and contraction processing is performed on the database according to the expansion and contraction quantity.

[0064] According to the above steps, after the expansion and contraction quantity is obtained, if the expansion and contraction quantity is positive, the database is expanded according to the expansion and contraction quantity; if the expansion and contraction quantity is negative, the database is contracted according to the expansion and contraction quantity.

[0065] The database expansion and contraction processing method of the embodiment can automatically perform the expansion and contraction processing on the database, and the whole process does not need manual participation, which not only can effectively save the labor cost, but also can save time and effort, has very fast response speed, and can effectively improve the efficiency of the expansion and contraction processing. In addition, since manual prediction is not needed, the accuracy is very high.

[0066] Figure 2 The flowchart of the second embodiment of the database expansion and contraction processing method of the application is shown in FIG. 2. Figure 2 The database expansion and contraction processing method of the embodiment can provide a more detailed technical solution on the basis of the above-mentioned embodiment. Figure 1 The database expansion and contraction processing method of the embodiment can specifically include the following steps. Figure 2

[0067] S200, the historical load data, the current load data, and the current size information of the database are obtained.​

[0068] S201, predicting first load data of the database at a future time t according to current load data, historical load data and a first load prediction model; the first load prediction model is a prediction model based on non-periodic change trend;

[0069] S202, predicting second load data of the database at the future time t according to the current load data, the historical load data and a second load prediction model; the second load prediction model is a prediction model based on periodic change trend;

[0070] S203, predicting third load data of the database at the future time t according to the current load data, the historical load data and a third load prediction model; the third load prediction model is a prediction model based on holiday or sudden event;

[0071] S204, accumulating the first load data, the second load data and the third load data of the database at the future time t as load data of the database at the future time t;

[0072] The steps S201-S203 of the embodiment are an implementation manner of the step S101 of the embodiment shown in the above Figure 1 The detailed implementation manner can refer to the description of the embodiment shown in the above Figure 1 The detailed implementation manner can refer to the description of the embodiment shown in the above

[0073] S205, detecting whether the current scale information and the scale-in / out configuration information corresponding to the load data at the future time t are included in the pre-configured scale-in / out configuration library; if yes, performing step S206; otherwise, if no, performing step S207;

[0074] S206, obtaining the scale-in / out quantity of the database at the future time t from the scale-in / out configuration library according to the current scale information and the load data at the future time t; and performing step S209;

[0075] S207, predicting the scale-in / out quantity of the database at the future time t according to the current scale information, the load data at the future time t and a pre-trained scale-in / out configuration model; and performing step S208;

[0076] S208, storing the load data at the future time t, the current scale information and the scale-in / out quantity of the database at the future time t predicted by the scale-in / out configuration model as a piece of scale-in / out configuration information in the scale-in / out configuration library; and performing step S209;

[0077] The step can enrich the information in the scale-in / out configuration library, so that the scale-in / out configuration library is more powerful.

[0078] S209, performing scaling processing on the database according to the scaling quantity.

[0079] It should be noted that the steps S208 and S209 can have no chronological relationship.

[0080] Further optionally, before the step S207 of predicting the scaling quantity of the database at the future time t according to the current scale information, the load data at the future time t, and the pre-trained scaling configuration model, the method can further include: training the scaling configuration model by using each piece of scaling configuration information in the scaling configuration library.

[0081] Further optionally, training the scaling configuration model by using each piece of scaling configuration information in the scaling configuration library can include the following steps:

[0082] (a2) inputting the load data and the scale information before scaling in each piece of scaling configuration information into the scaling configuration model to obtain the scaling quantity predicted by the scaling configuration model;

[0083] In this embodiment, each piece of scaling configuration information in the scaling configuration library is used to train the scaling configuration model, and each piece of scaling configuration information corresponds to a training data. During training, the load data and the scale information before scaling in the scaling configuration information are input information, and the predicted scaling quantity output by the scaling configuration model can be obtained.

[0084] (b2) applying the predicted scaling quantity to the mirror database corresponding to the load data and the scale information before scaling to obtain the Transactions Per Second (TPS) of the mirror database;

[0085] In the training of this embodiment, the corresponding configuration information is not used to adjust the parameters of the predicted scaling quantity and the scaling quantity, of course, in actual application, the predicted scaling quantity and the scaling quantity in the corresponding configuration information can also be used to adjust the parameters, so that the predicted scaling quantity output by the scaling configuration model is always consistent with the scaling quantity in the corresponding configuration information, the parameters of the scaling configuration model are determined, and then the scaling configuration model is determined.

[0086] In this embodiment, by applying the predicted scaling quantity to the mirror database corresponding to the load data and the scale information before scaling, the performance of the mirror database is detected to determine whether the prediction is accurate, and if not, the parameters of the scaling configuration model are adjusted. For example, in this embodiment, the TPS of the mirror database is detected, and in actual application, other parameters can also be used, which will not be repeated here.

[0087] (c2) judging whether the TPS reaches a preset maximum threshold value in continuous preset times of training; if not, performing step (d2); if yes, performing step (e2);

[0088] The preset maximum threshold value in the embodiment can be set according to actual needs, which is not limited herein.

[0089] (d2) adjusting parameters of the scale-out / in configuration model; returning to step (a2) to input the next scale-out / in configuration information into the scale-out / in configuration model to continue training;

[0090] (e2) judging whether the TPS reaches a preset maximum threshold value in continuous preset times of training; if yes, the training is completed, parameters of the scale-out / in configuration model are determined, and then the scale-out / in configuration model is determined; otherwise, returning to step (a2) to input the next scale-out / in configuration information into the scale-out / in configuration model to continue training.

[0091] The scale-out / in method of the database in the embodiment can automatically perform scale-out / in processing on the database, and the whole process does not need human participation, which not only can effectively save labor cost, but also can save time and effort, has very fast response speed, and can effectively improve the efficiency of scale-out / in processing. In addition, since human prediction is not needed, the accuracy is very strong.

[0092] Figure 3 The structure diagram of the database scale-out / in processing device embodiment one of the application is shown in FIG. 1. Figure 3 As shown in FIG. 1, the database scale-out / in processing device in the embodiment can specifically include:

[0093] The data acquisition module 10 is configured to acquire historical load data, current load data and current scale information of the database.

[0094] The prediction module 11 is configured to predict load data of the database at a future time t according to the current load data, the historical load data and a pre-trained load prediction model acquired by the data acquisition module 10.

[0095] The scale-out / in quantity acquisition module 12 is configured to acquire a scale-out / in quantity of the database at the future time t according to the current scale information acquired by the data acquisition module 10 and the load data at the future time t predicted by the prediction module 11.

[0096] The scale-out / in processing module 13 is configured to perform scale-out / in processing on the database according to the scale-out / in quantity acquired by the scale-out / in quantity acquisition module 12.

[0097] The implementation principle and technical effect of the database scale processing device of the embodiment are the same as those of the above-mentioned related method embodiments, and details can be referred to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0098] Further optionally, the above Figure 3 In the database scale device of the embodiment shown, the prediction module 11 is specifically configured to:

[0099] predict first load data of the database at a future time t according to the current load data, the historical load data and a first load prediction model; the first load prediction model is a prediction model based on non-periodic change trend; predict second load data of the database at the future time t according to the current load data, the historical load data and a second load prediction model; the second load prediction model is a prediction model based on periodic change trend; and / or predict third load data of the database at the future time t according to the current load data, the historical load data and a third load prediction model; the third load prediction model is a prediction model based on holidays or emergencies;

[0100] accumulate the first load data, the second load data and / or the third load data of the database at the future time t as the load data of the database at the future time t.

[0101] Further optionally, the above Figure 3 In the database scale device of the embodiment shown, the scale quantity obtaining module 12 is specifically configured to:

[0102] obtain the scale quantity of the database at the future time t from the scale configuration library according to the current scale information and the load data at the future time t; wherein each piece of scale configuration information in the scale configuration library includes a corresponding relationship among load data, scale information before scale and scale quantity.

[0103] Alternatively, the scale quantity obtaining module 12 is specifically configured to:

[0104] predict the scale quantity of the database at the future time t according to the current scale information, the load data at the future time t and a pre-trained scale configuration model.

[0105] Figure 4 is a structural diagram of the second embodiment of the database scale processing device of the present application. As Figure 4 shown, the database scale processing device of the embodiment, on the basis of the above Figure 3 embodiment, further introduces the technical scheme of the present application in more detail.

[0106] As Figure 4As shown, the database scale-out processing device of the embodiment further includes:

[0107] The detection module 14 is configured to detect and determine whether the scale-out configuration library includes scale-out configuration information corresponding to the current scale information and the load data at the future time t; and / or

[0108] The storage module 15 is configured to store the load data at the future time t, the current scale information, and the scale-out quantity of the database at the future time t predicted by the scale-out configuration model as a piece of scale-out configuration information in the scale-out configuration library.

[0109] Correspondingly, when the scale-out quantity acquisition module 12 needs to acquire the scale-out quantity corresponding to the load data at the future time t and the current scale information in the future, the scale-out configuration information can be directly acquired from the scale-out configuration library processed by the storage module 15, and then the scale-out quantity corresponding to the load data at the future time t and the current scale information can be acquired.

[0110] Figure 4 In the embodiment shown, the detection module 14 and the storage module 15 are taken as an example. In a specific application scenario, the detection module 14 can first detect whether the scale-out configuration library includes scale-out configuration information corresponding to the current scale information and the load data at the future time t; if the detection module 14 determines that the scale-out configuration library includes the scale-out configuration information, the scale-out quantity acquisition module 12 is triggered to acquire the scale-out quantity of the database at the future time t from the scale-out configuration library according to the current scale information and the load data at the future time t; otherwise, if the detection module 14 determines that the scale-out configuration library does not include the scale-out configuration information, the scale-out quantity acquisition module 12 is triggered to predict the scale-out quantity of the database at the future time t according to the current scale information, the load data at the future time t, and the pre-trained scale-out configuration model.

[0111] Correspondingly, after the scale-out quantity acquisition module 12 predicts the scale-out quantity of the database at the future time t according to the current scale information, the load data at the future time t, and the pre-trained scale-out configuration model, the storage module 15 can be triggered to store the load data at the future time t, the current scale information, and the scale-out quantity of the database at the future time t predicted by the scale-out configuration model as a piece of scale-out configuration information in the scale-out configuration library.

[0112] Further optionally, as Figure 4 As shown, the database scale-out processing device of the embodiment further includes:

[0113] The training module 16 is configured to train the scale-out configuration model by using each piece of scale-out configuration information in the scale-out configuration library.

[0114] Correspondingly, the scaling up / down quantity acquisition module 12 is used to predict the scaling up / down quantity of the database at future time t based on the current scale information, the load data at future time t, and the scaling up / down configuration model pre-trained by the training module 16.

[0115] Further, optionally, the training module 16 is used for:

[0116] Input the load data and scale information before scaling up / down from each scaling up / down configuration information into the scaling up / down configuration model to obtain the scaling up / down quantity predicted by the scaling up / down configuration model.

[0117] The predicted scaling up / down quantities are applied to the mirror database corresponding to the load data and the scale information before scaling up / down, and the TPS of the mirror database is obtained.

[0118] Determine whether the TPS reaches the preset maximum threshold in a series of preset training sessions.

[0119] If not, adjust the parameters of the scaling configuration model;

[0120] Repeat the above steps, using each scaling configuration information to continuously train the scaling configuration model until the TPS reaches the preset maximum threshold in a preset number of consecutive training sessions. Then determine the parameters of the scaling configuration model and thus determine the scaling configuration model.

[0121] The database scaling processing device in this embodiment achieves the same implementation principle and technical effect as the above-mentioned related method embodiments by using the above-mentioned modules. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0122] Figure 5 This is a structural diagram of an embodiment of the computer device of the present invention. Figure 5 As shown, the computer device of this embodiment includes: one or more processors 30, and a memory 40. The memory 40 is used to store one or more programs. When the one or more programs stored in the memory 40 are executed by one or more processors 30, the one or more processors 30 perform the above-described actions. Figures 1-2 The database scaling method shown in the embodiment. Figure 5 The embodiment shown takes a plurality of processors 30 as an example.

[0123] For example, Figure 6 An example diagram of a computer device provided by the present invention. Figure 6 A block diagram of an exemplary computer device 12a suitable for implementing embodiments of the present invention is shown. Figure 6 The computer device 12a shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0124] like Figure 6 As shown, computer device 12a is represented in the form of a general-purpose computing device. Components of computer device 12a may include, but are not limited to: one or more processors 16a, system memory 28a, and bus 18a connecting different system components (including system memory 28a and processor 16a).

[0125] Bus 18a represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0126] Computer device 12a typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12a, including volatile and non-volatile media, removable and non-removable media.

[0127] System memory 28a may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30a and / or cache memory 32a. Computer device 12a may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34a may be used to read and write non-removable, non-volatile magnetic media (…). Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18a via one or more data media interfaces. System memory 28a may include at least one program product having a set (e.g., at least one) of program modules configured to execute the invention described above. Figures 1-4 Functions of each embodiment.

[0128] Program / utility 40a having a set of program modules 42a can be stored in system memory 28a and implemented by computer 12a. In this embodiment, program modules 42a include, but are not limited to, operating system 44a, one or more application programs 46a, other program modules 48a, and program data 50a. Each of the operating system 44a, one or more application programs 46a, other program modules 48a, and program data 50a may Figures 1-4 The functions and / or methods of each embodiment.

[0129] Computer 12a can also communicate with one or more external devices 14a such as a keyboard or pointing device, a display 24a, etc.; one or more devices that enable a user to interact with computer 12a; and / or one or more devices that enable computer 12a to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interfaces 22a. Still yet, computer 12a can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 20a. As depicted, network adapter 20a communicates with the other components of computer 12a via bus 18a. It should be appreciated that although not shown, other hardware and / or software components could be used in conjunction with computer 12a. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0130] Processor 16a performs a variety of functions as described above by executing program code stored in system memory 28a.

[0131] The present application also provides a computer readable medium having stored thereon a computer program, which, when executed by a processor, implements the database scale-out / in processing method as shown in the above embodiments.

[0132] The computer readable medium of the present embodiment can include the RAM 30a in the system memory 28a, and / or the cache memory 32a, and / or the storage system 34a in the embodiments shown above. Figure 6 The system memory 28a in the embodiments shown above.

[0133] With the development of technology, the dissemination of computer programs is no longer limited to tangible media, but can also be downloaded directly from the network or obtained in other ways. Therefore, the computer readable medium in the present embodiment can include not only tangible media, but also intangible media.

[0134] The computer readable medium of the present embodiment can employ any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present document, a computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.

[0135] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein. The propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0136] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0137] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In an embodiment, the present application is directed to computer program products comprising machine-readable media for carrying or having machine-executable instructions or programs thereon.

[0138] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or diverse functions can be combined or integrated into another system provided that the purpose of the embodiments can be achieved.

[0139] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0140] In addition, each functional unit in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware, or in the form of hardware plus software function units.

[0141] The integrated unit implemented in the form of software function units can be stored in a computer readable storage medium. The software function unit stored in the storage medium includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a variety of media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0142] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for scaling up and down a database, characterized in that, The method includes: Retrieve historical load data, current load data, and current scale information from the database; Based on the current load data, the historical load data, and the pre-trained load prediction model, predict the load data of the database at a future time t; Based on the current scale information and the load data at the future time t, obtain the number of times the database will be expanded or shrunk at the future time t; The database is scaled up or down according to the specified scaling requirements; Based on the current scale information and the load data at the future time t, obtain the scaling up / down quantity of the database at the future time t, including: Based on the current scale information, the load data at the future time t, and the pre-trained scaling configuration model, predict the scaling up or down quantity of the database at the future time t. The training of the scaling configuration model includes: The load data and scale information before scaling up or down from each scaling up / down configuration in the scaling up / down configuration library are input into the scaling up / down configuration model to obtain the scaling up / down quantity predicted by the scaling up / down configuration model. The predicted scaling up / down quantities are applied to the mirror database corresponding to the load data and the scale information before scaling up / down, and the TPS of the mirror database is obtained. Determine whether the TPS reaches the preset maximum threshold in a series of preset training sessions. If not, adjust the parameters of the scaling configuration model.

2. The method according to claim 1, characterized in that, Based on the current load data, the historical load data, and the pre-trained load prediction model, predict the load data of the database at a future time t, including: Based on the current load data, the historical load data, and the first load prediction model, predict the first load data of the database at a future time t; the first load prediction model is a prediction model based on a non-periodic variation trend. Based on the current load data, the historical load data, and the second load prediction model, predict the second load data of the database at a future time t; the second load prediction model is a prediction model based on a periodic variation trend. And / or based on the current load data, the historical load data, and the third load prediction model, predict the third load data of the database at a future time t; the third load prediction model is a prediction model based on holidays or sudden events. The first load data, the second load data, and / or the third load data of the database at the future time t are summed to form the load data of the database at the future time t.

3. The method according to claim 1, characterized in that, Based on the current scale information, the load data at the future time t, and the pre-trained scaling configuration model, before predicting the scaling up or down quantity of the database at the future time t, the method further includes: The system detects and determines that the scaling configuration library does not include the current scale information and the scaling configuration information corresponding to the load data at the future time t. And / or, based on the current scale information, the load data at the future time t, and a pre-trained scaling configuration model, after predicting the scaling-up / scaling requirement of the database at the future time t, the method further includes: The load data at the future time t, the current scale information, and the number of times the database will be scaled up or down as predicted by the scaling configuration model at the future time t are stored as a scaling configuration information in the scaling configuration library.

4. A database scaling processing device, characterized in that, The device includes: The data acquisition module is used to acquire historical load data, current load data, and current scale information of the database. The prediction module is used to predict the load data of the database at a future time t based on the current load data, the historical load data, and the pre-trained load prediction model. The scaling up / down quantity acquisition module is used to acquire the scaling up / down quantity of the database at the future time t based on the current scale information and the load data at the future time t. The scaling up / down processing module is used to perform scaling up / down processing on the database according to the scaling up / down quantity; The expansion / shrinkage quantity acquisition module is used for: Based on the current scale information, the load data at the future time t, and the pre-trained scaling configuration model, predict the scaling up or down quantity of the database at the future time t. It also includes: a training module, used for: The load data and scale information before scaling up or down from each scaling up / down configuration in the scaling up / down configuration library are input into the scaling up / down configuration model to obtain the scaling up / down quantity predicted by the scaling up / down configuration model. The predicted scaling up / down quantities are applied to the mirror database corresponding to the load data and the scale information before scaling up / down, and the TPS of the mirror database is obtained. Determine whether the TPS reaches the preset maximum threshold in a series of preset training sessions. If not, adjust the parameters of the scaling configuration model.

5. The apparatus according to claim 4, characterized in that, The prediction module is used for: Based on the current load data, the historical load data, and the first load prediction model, predict the first load data of the database at a future time t; the first load prediction model is a prediction model based on a non-periodic variation trend. Based on the current load data, the historical load data, and the second load prediction model, predict the second load data of the database at a future time t; the second load prediction model is a prediction model based on a periodic variation trend. And / or based on the current load data, the historical load data, and the third load prediction model, predict the third load data of the database at a future time t; the third load prediction model is a prediction model based on holidays or sudden events. The first load data, the second load data, and / or the third load data of the database at the future time t are summed to form the load data of the database at the future time t.

6. The apparatus according to claim 4, characterized in that, The device further includes: The detection module is used to detect and determine that the scaling configuration library does not include the current scale information and the scaling configuration information corresponding to the load data at the future time t; and / or The storage module is used to store the load data at the future time t, the current scale information, and the number of times the database will be expanded or scaled at the future time t as predicted by the scaling configuration model, as a scaling configuration information in the scaling configuration library.

7. A computer device, characterized in that, The device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-3.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-3.

Images