Database management method, computer device, nonvolatile readable storage medium and program product
By building a business volume prediction model and performing real-time parameter analysis, the scaling up and down of database service instances is dynamically adjusted, solving the problem of insufficient or excessive resources in cloud databases and optimizing the cost and reliability of cloud database services.
Patent Information
- Application Number
- PCT/CN2025/098125
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-05-29
- Filing Date
- 2025-05-29
- Publication Date
- 2025-12-04
AI Technical Summary
In existing cloud database services, users cannot accurately assess the correlation between business load and instance specifications, leading to service outages when resources are insufficient and increased costs when resources are abundant, making it impossible to respond to peak loads in a timely manner.
Based on historical time-series business data from the database, a business volume prediction model is constructed. The business volume parameters are decomposed through a time series prediction algorithm, and combined with real-time parameter information, the number of database service instances that can be scaled up or down is dynamically adjusted to respond to changes in business volume.
It enables the prediction of scaling up and down for future time periods, reduces cold start latency, optimizes the cost and reliability of database services, and improves the performance and availability of database services.
Smart Images

Figure CN2025098125_04122025_PF_FP_ABST
Abstract
Description
Database management method, computer device, non-volatile readable storage medium and program product
[0001] Cross-reference to related applications
[0002] The present application claims priority to the Chinese patent application No. 202410675740.9, filed on May 29, 2024, entitled "Database management method, computer device, storage medium and program product", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD
[0003] The present application relates to the field of cloud computing technology, and in particular, to a database management method, a computer device, a non-volatile readable storage medium and a program product. BACKGROUND
[0004] With the continuous development of cloud computing, the database technology presents a trend from traditional centralized to distributed development. The current stage of cloud database development is mainly in the stage of "DB on Cloud", that is, when creating a database instance, the user determines the computing specifications and storage specifications of the database instance and other resources in advance based on the business load peak. This database service providing method has the following problems:
[0005] 1. The user cannot accurately associate the business load with the instance specifications (or the business load cannot be evaluated at all). If there is a lack of resources, only emergency specification changes can be made, which cannot respond to the load peak in time, and there will be a short service outage during the specification change, and the user experience will be poor.
[0006] 2. In order to reduce the frequency of service interruption caused by specification changes, the user often applies for corresponding resources according to the peak value within a certain period of time, and then the corresponding peak value resources will be continuously charged. However, most businesses are in peak time for a short period of time, resulting in unnecessary resources being paid for most of the time, ultimately leading to an increase in user usage costs. SUMMARY
[0007] Therefore, it is necessary to provide a database management method, device, computer device and non-volatile readable storage medium capable of improving the reliability and continuity of database services to solve the above technical problems.
[0008] In a first aspect, a database management method is provided, the method comprising:
[0009] determining a historical traffic parameter based on historical time-series business data of the database, the historical traffic parameter being used to describe a traffic development law;
[0010] According to the historical traffic parameter, a traffic prediction model is constructed, and a traffic prediction result in a target time period is determined based on the traffic prediction model;
[0011] Based on the real-time parameter information of the database, the service bearing pressure of the database is determined, and the real-time parameter information is used to describe the running state of the database;
[0012] Based on the traffic prediction result and / or the service bearing pressure of the database, a target scale-in / out quantity corresponding to the database service instance is determined, and the database service instance is used to provide database service;
[0013] Based on the target scale-in / out quantity, the database is scaled in / out to respond to the database distribution request.
[0014] In some embodiments, based on the historical time series traffic data of the database, the historical traffic parameter is determined, including:
[0015] The historical time series traffic data of the database is obtained, and based on the historical time series traffic data, feature information of the historical time series traffic data is determined, and the feature information at least includes time characteristics and traffic characteristics;
[0016] Based on the feature information of the historical time series traffic data, a historical traffic curve is constructed;
[0017] Based on a time series prediction algorithm, the historical traffic curve is decomposed to obtain the traffic parameter.
[0018] In some embodiments, according to the historical traffic parameter, the traffic prediction model is constructed, including:
[0019] According to the historical traffic parameter, a traffic parameter model represented by a target function is constructed;
[0020] The traffic parameter model is fitted to construct the traffic prediction model.
[0021] In some embodiments, the historical traffic parameter at least includes a trend component, a periodic component, a holiday parameter and a residual component, and according to the historical traffic parameter, the traffic parameter model represented by the target function is constructed, including:
[0022] Based on the first target function and the trend component, a trend component model is constructed, and the first target function includes: T(t) = (1-α)T(t-1)
[0023] Wherein, T(t) represents the trend component at time t, t represents time, ɑ∈[0,1], represents a trend parameter, and T(t-1) represents the trend component at time t-1;
[0024] Based on the second objective function and the periodic component, a periodic component model is constructed, and the second objective function comprises: C(t) = (1 - β)C(t - k)
[0025] wherein C(t) represents the periodic component at time t, k ∈ [0, 1] represents the period length, β represents the period parameter, and C(t - k) represents the trend component at time t - k;
[0026] Based on the third objective function and the holiday parameter, a holiday parameter model is constructed, and the third objective function comprises:
[0027] wherein H(t) represents the holiday parameter at time t, a ∈ [0, 10000] represents the coefficient affected by the holiday, and T(t - 1) represents the trend component at time t - 1;
[0028] Based on the fourth objective function and the residual component, a residual component model is constructed, and the fourth objective function comprises: R(t) = Y(t - 1) - H(t - 1) * T(t - 1) * C(t - 1)
[0029] wherein R(t) represents the residual component at time t, Y(t - 1) represents the traffic volume prediction value at time t - 1, H(t - 1) represents the holiday parameter at time t - 1, and C(t - 1) represents the periodic component at time t - 1.
[0030] In some embodiments, fitting the traffic volume parameter model to construct a traffic volume prediction model comprises:
[0031] The first objective function, the second objective function, and the third objective function are multiplied and the fourth objective function is accumulated to construct the traffic volume prediction model, comprising: Y(t) = C(t) × H(t) × T(t) + R(t)
[0032] wherein Y(t) represents the traffic volume prediction value at time t.
[0033] In some embodiments, based on the traffic volume prediction model, determining the traffic volume prediction result in the target time period comprises:
[0034] Setting a time period;
[0035] Based on the time period, determining the target time period corresponding to the current time;
[0036] Based on the traffic volume prediction model, at least one traffic volume prediction value in the target time period is obtained to generate the traffic volume prediction result in the target time period.
[0037] In some embodiments, the method further comprises determining whether the database needs to be scaled based on the traffic volume prediction result, comprising:
[0038] obtaining a quantity of database service instances at a current time, and determining a service carrying capacity based on the quantity of database service instances at the current time;
[0039] in response to detecting that a difference between the service carrying capacity and the service quantity prediction value is greater than a first preset threshold, determining that the database needs to be scaled out;
[0040] in response to detecting that the difference between the service carrying capacity and the service quantity prediction value is less than a second preset threshold, determining that the database needs to be scaled in.
[0041] In some embodiments, based on the determination result of whether the database needs to be scaled out or in, determining the first scale-out / in quantity of the database service instances comprises:
[0042] in response to detecting that the database needs to be scaled out or in, determining the first scale-out / in quantity of the database service instances based on an absolute value of the difference between the service carrying capacity and the service quantity prediction value, and the service carrying capacity of each database service instance.
[0043] In some embodiments, determining the service carrying pressure of the database based on the real-time parameter information of the database comprises:
[0044] obtaining real-time parameter information and performing normalization processing on the real-time parameter information, the real-time parameter information comprising at least one of: resource utilization of the database service instance, request response time delay, and waiting queue length;
[0045] fitting the normalized real-time parameter information using a first target fitting function to obtain a first target fitting value, the first target fitting function comprising: A = aw1 + bw2 + cw3
[0046] wherein A represents the first target fitting value, w1, w2, and w3 each represent a weight coefficient, a represents the resource utilization of the database service instance, b represents the request response time delay, and c represents the waiting queue length;
[0047] determining the service carrying pressure of the database based on the first target fitting value.
[0048] In some embodiments, determining the service carrying pressure of the database based on the first target fitting value comprises:
[0049] determining the service carrying pressure of the database corresponding to the first target fitting value based on a first mapping relationship between the first target fitting value and the service carrying pressure of the database.
[0050] In some embodiments, determining whether the database needs to be scaled out or in based on the service carrying pressure of the database comprises:
[0051] In response to detecting that the service load pressure of the database is greater than a third preset threshold, it is determined that the database needs to be scaled out;
[0052] In response to detecting that the service load pressure of the database is less than a fourth preset threshold, it is determined that the database needs to be scaled in.
[0053] In some embodiments, in response to detecting that the database needs to be scaled out / in, a second scaling number of the database service instance is calculated and determined, including:
[0054] Obtaining a scaling threshold corresponding to real-time parameter information, the real-time parameter information at least including resource usage, request response time delay, the resource usage at least including central processing unit usage, memory usage;
[0055] Using a second target fitting function, a second target fitting value is calculated and determined, and the second target fitting value is defined as the second scaling number of the database service instance.
[0056] In some embodiments, the second target fitting function includes:
[0057] Wherein, D(t) represents the second target fitting value, w4, w5, w6 all represent weight coefficients, cpu usage(t) represents the central processing unit usage, cpu usage(s) represents the central processing unit scaling threshold, mem usage(t) represents the memory usage, mem usage(s) represents the memory scaling threshold, qps delay(t) represents the request response time delay, qps delay(s) represents the request response time delay scaling threshold, and t represents time.
[0058] In some embodiments, based on the service volume prediction result and / or the service load pressure of the database, a target scaling number corresponding to the database service instance is determined, including:
[0059] Obtaining a first scaling number determined based on the service volume prediction result, and a second scaling number determined based on the service load pressure of the database;
[0060] In response to detecting that the current time is within a first target time period, the first scaling number is determined as the target scaling number of the database service instance within a second target time period, the first target time period being a time period in which scaling has been performed according to the service volume prediction result, and the second target time period being a future time period in which scaling has not been performed;
[0061] Based on the second scaling number, the target scaling number of the database service instance at the current time is determined.
[0062] In some embodiments, determining the target scale-out number of the database service instance at the current time based on the second scale-out number comprises:
[0063] In response to detecting that the current time needs to be scaled out, defining the second scale-out number as the target scale-out number of the database service instance at the current time;
[0064] In response to detecting that the current time needs to be scaled in, obtaining a target value of the scale-out amount in a third target time period, the third target time period being a previous time period of the time period in which the scale-out has been performed;
[0065] In response to detecting that the target value is less than a fifth preset threshold, defining the second scale-out number as the target scale-out number of the database service instance at the current time.
[0066] In some embodiments, the method of calculating the target value comprises:
[0067] Obtaining the scale-out numbers corresponding to the plurality of time nodes in the third target time period, and sorting the scale-out numbers in the order of time from far to near;
[0068] Fitting the scale-out numbers corresponding to the plurality of time nodes by using a third target fitting function to obtain the target value, wherein the third target fitting function comprises: B = (x1ε1+x2ε2+...+xnεn) / n n ε n ) / n
[0069] Wherein B represents the target value, X = x1, x2,..., xn represents the scale-out numbers corresponding to the 1st, 2nd,..., nth time nodes, Y = ε1, ε2,..., εn represents the weight coefficients corresponding to each scale-out amount, and n represents the number of time nodes. n n The greater the n, the greater the weight coefficient. In some embodiments, the greater the n, the greater the weight coefficient.
[0070] In some embodiments, based on the target scale-out number, the method of scaling out / in the database comprises:
[0071] Detecting whether the current time database is in a scale-out / in cooling period;
[0072] If not in the scale-out / in cooling period, scaling out / in the database based on the target scale-out number.
[0073] In some embodiments, before responding to the database distribution request, the method further comprises:
[0074] Obtaining the identification information carried by the database distribution request;
[0075] Based on the identification information, determining the historical connection record of the database distribution request initiator and the target database service instance;
[0076] forwarding the database distribution request to the target database service instance in response to detecting that the historical connection record exists and the target database service instance still exists;
[0077] in response to detecting that the historical connection record does not exist and / or the target database service instance does not exist, determining a database service instance for forwarding the database distribution request according to a service carrying pressure of each database service instance.
[0078] In some embodiments, in response to the database distribution request, the method further comprises:
[0079] obtaining relevant information of the database distribution request, the relevant information at least including a request arrival time, a request forwarding time and a request return time;
[0080] based on the relevant information, calculating a request response time delay.
[0081] In some embodiments, the method further comprises:
[0082] if no database distribution request is detected within a preset time range, scaling down the number of database service instances to a preset value.
[0083] In a second aspect, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:
[0084] based on historical time-series service data of the database, determining a historical service volume parameter, the historical service volume parameter being used to describe a service volume development law;
[0085] based on the historical service volume parameter, constructing a service volume prediction model, and based on the service volume prediction model, determining a service volume prediction result in a target time period;
[0086] based on real-time parameter information of the database, determining a service carrying pressure of the database, the real-time parameter information being used to describe a database running state;
[0087] based on the service volume prediction result and / or the service carrying pressure of the database, determining a target scaling number corresponding to a database service instance, the database service instance being used to provide a database service;
[0088] based on the target scaling number, scaling the database to respond to a database distribution request.
[0089] In a third aspect, a non-volatile computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:
[0090] Based on the historical time-series business data of the database, a historical traffic parameter is determined, the historical traffic parameter being used to describe a traffic development law;
[0091] According to the historical traffic parameter, a traffic prediction model is constructed, and based on the traffic prediction model, a traffic prediction result in a target time period is determined;
[0092] Based on real-time parameter information of the database, a service bearing pressure of the database is determined, the real-time parameter information being used to describe a running state of the database;
[0093] Based on the traffic prediction result and / or the service bearing pressure of the database, a target scale quantity corresponding to a database service instance is determined, the database service instance being used to provide a database service;
[0094] Based on the target scale quantity, the database is scaled to respond to a database distribution request.
[0095] In a fourth aspect, a computer program product is provided, the computer program product comprising a computer program, the computer program being executed by a processor to implement the following steps:
[0096] Based on the historical time-series business data of the database, a historical traffic parameter is determined, the historical traffic parameter being used to describe a traffic development law;
[0097] According to the historical traffic parameter, a traffic prediction model is constructed, and based on the traffic prediction model, a traffic prediction result in a target time period is determined;
[0098] Based on real-time parameter information of the database, a service bearing pressure of the database is determined, the real-time parameter information being used to describe a running state of the database;
[0099] Based on the traffic prediction result and / or the service bearing pressure of the database, a target scale quantity corresponding to a database service instance is determined, the database service instance being used to provide a database service;
[0100] Based on the target scale quantity, the database is scaled to respond to a database distribution request.
[0101] The database management method, computer device, non-volatile readable storage medium and program product, the method comprises: determining a historical traffic parameter based on historical time-series traffic data of a database, the historical traffic parameter being used to describe a traffic development law; constructing a traffic prediction model according to the historical traffic parameter, and determining a traffic prediction result in a target time period based on the traffic prediction model; determining a traffic bearing pressure of the database based on real-time parameter information of the database, the real-time parameter information being used to describe a database running state; determining a target expansion or shrinkage quantity corresponding to a database service instance based on the traffic prediction result and / or the traffic bearing pressure of the database, the database service instance being used to provide a database service; and expanding or shrinking the database based on the target expansion or shrinkage quantity to respond to a database distribution request. The application can predict expansion or shrinkage in a future time period and perform expansion or shrinkage in advance, thereby reducing cold start delay. Based on this, the expansion or shrinkage result is corrected based on real-time traffic bearing pressure to meet business performance requirements, thereby reducing the cost of providing a database service and improving the performance and availability of the database service. BRIEF DESCRIPTION OF DRAWINGS
[0102] Fig. 1 is a diagram of an application environment of a database management method in an embodiment;
[0103] Fig. 2 is a flowchart of a database management method in an embodiment;
[0104] Fig. 3 is another flowchart of a database management method in an embodiment;
[0105] Fig. 4 is a diagram of working principles of a database service agent module of a database management method in an embodiment;
[0106] Fig. 5 is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION
[0107] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.
[0108] It should be understood that, in the description of the present application, unless the context clearly requires otherwise, the terms "comprise", "comprising", and the like in the entire specification mean the inclusion of the elements recited therein but not to the exclusion of other elements or the addition of other elements. That is, they are "including but not limited to".
[0109] It should also be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0110] It should be noted that the terms "S1," "S2," etc., are used only for descriptive purposes and do not specifically refer to the order or sequence, nor are they intended to limit this application. They are merely for the convenience of describing the method of this application and should not be construed as indicating the sequential order of the steps. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0111] The database management method provided in this application can be applied to the application environment shown in Figure 1. The terminal 102 communicates with a data processing platform set up on the server 104 via a network. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server 104 can be a standalone server or a server cluster consisting of multiple servers.
[0112] In one embodiment, as shown in Figure 2, a database management method is provided. Taking the application of this method to the terminal in Figure 1 as an example, the method includes the following steps:
[0113] S1: Based on historical time-series business data from the database, determine historical business volume parameters, which are used to describe the development pattern of business volume.
[0114] It should be noted that, as shown in Figure 3, historical time-series business data is stored in a time-series database, which represents a set of data with timestamps arranged in chronological order. This business data can be data from the server's log files. The historical business volume parameters can include trend components (the trend of business data changes in non-periodic areas), periodic components (most businesses are periodic on a daily, weekly, or monthly basis), holiday parameters (businesses are mostly related to holidays), and residual components (the remaining terms that the model cannot fit, which can be considered to conform to a normal distribution).
[0115] In some optional implementations, determining historical traffic volume parameters based on historical time-series business data from the database includes:
[0116] Obtain historical time-series business data from the database, and based on the historical time-series business data, determine the characteristic information of the historical time-series business data. The characteristic information includes at least time characteristics and business characteristics. The time characteristics are the time nodes, and the business characteristics can be the business volume corresponding to each time node.
[0117] Based on the feature information of historical time-series business data, a historical business volume curve is constructed. That is, the time node can be defined as the horizontal axis of the curve, and the business volume can be defined as the vertical axis of the curve, thereby generating a historical business volume curve.
[0118] To ensure suitability for various business scenarios, the historical business volume curve is decomposed based on a time series forecasting algorithm to obtain business volume parameters. The time series forecasting algorithm is the Prophet algorithm, which is used to decompose the value to be predicted into multiple components, such as trend components, periodic components, holiday parameters, and residual components.
[0119] In the above implementation, historical business volume parameters are obtained and decomposed to describe the development pattern of business volume, which is then used to build a business volume prediction model and improve the accuracy of the model construction.
[0120] S2: Based on historical business volume parameters, construct a business volume prediction model, and based on the business volume prediction model, determine the business volume prediction results for the target time period.
[0121] It should be noted that the target time period is a future time period corresponding to the current moment. The time period range can be set according to actual needs. For example, if the current moment is 3:00, and the time period to which the current moment belongs is from 3:01 to 3:30, with a time period range of 30 minutes, then the target time period can be from 3:31 to 4:00, and so on. In addition, this business volume prediction model is stored in the business pressure controller (Pressure Predictor). The business pressure controller performs data fitting analysis based on the time series data recorded in the database service agent module, and predicts the business volume of the next database service based on the database service access trend, so as to initiate scaling requests in advance, thereby reducing the cold start latency of service startup. In addition, the business replication controller drives the file system backup agent to access the file system of the container running the database service and initiates data scanning to back up incremental business data to the relay object storage. The business restore controller periodically drives the file system restore agent to copy the data on the relay object storage to the file system of the database instance in the backup data center, realizing differential replication of business data and completing data synchronization.
[0122] In some optional implementations, constructing a traffic volume prediction model based on historical traffic volume parameters includes:
[0123] Based on historical traffic volume parameters, a traffic volume parameter model based on an objective function is constructed, wherein the objective function may include a first objective function, a second objective function, a third objective function, and a fourth objective function;
[0124] Fit the business volume parameter model to construct a business volume prediction model.
[0125] In some optional implementations, constructing a traffic volume parameter model based on an objective function representation, according to historical traffic volume parameters, includes:
[0126] Based on the first objective function and the trend component, a trend component model is constructed. The first objective function includes: T(t)=(1-α)T(t-1)
[0127] Where T(t) represents the trend component at time t, t represents time, α∈[0,1] represents the trend parameter, T(t-1) represents the trend component at time t-1, and the predicted business data at time t includes the trend component values of the previous business data.
[0128] Based on the second objective function and the periodic component, a periodic component model is constructed. The second objective function includes: C(t)=(1-β)C(tk)
[0129] Where C(t) represents the periodic component at time t, k∈[0,1] represents the period length, β represents the period parameter, C(tk) represents the trend component at time tk, and the predicted business data includes the periodic component values of the previous business data.
[0130] Based on the third objective function and holiday parameters, a holiday parameter model is constructed. The third objective function includes:
[0131] Where H(t) represents the holiday parameter at time t, and a∈[0,10000] represents the coefficient affected by the holiday;
[0132] Based on the fourth objective function and the residual components, a residual component model is constructed. The fourth objective function includes: R(t)=Y(t-1)-H(t-1)*T(t-1)*C(t-1)
[0133] Where R(t) represents the residual component at time t, Y(t-1) represents the predicted business volume at time t-1, H(t-1) represents the holiday parameter at time t-1, and C(t-1) represents the periodic component at time t-1. The residual component is the difference between the actual value and the predicted value. Increasing the residual improves the fitting quality of the model.
[0134] In some alternative implementations, fitting the traffic volume parameter model to construct a traffic volume prediction model includes:
[0135] The first, second, and third objective functions are multiplied together, and the fourth objective function is summed to construct a business volume prediction model, including: Y(t)=C(t)×H(t)×T(t)+R(t)
[0136] Where Y(t) represents the predicted traffic volume at time t.
[0137] In some optional implementations, determining the traffic volume forecast results for the target time period based on the traffic volume forecasting model includes:
[0138] Set the time period, which is the value mentioned above that can be set according to actual needs, such as 30 minutes;
[0139] Based on the time cycle, determine the target time period corresponding to the current moment. For example, if the current moment is 3 o'clock, and the time period to which the current moment belongs is from 3:01 to 3:30, and the time cycle range is 30 minutes, then the target time period can be from 3:31 to 4:00, and so on.
[0140] Based on the business volume prediction model, at least one business volume prediction value is obtained within the target time period to generate the business volume prediction result within the target time period. The target time period generally includes multiple time points and their corresponding business volume prediction values.
[0141] In the above implementation, the business volume prediction model can predict the business volume of the database in advance, thereby determining whether to scale up or down based on the business volume, so as to issue scaling up or down requests in advance and reduce the response latency of the requests.
[0142] In some alternative implementations, the method further includes determining whether the database needs to be scaled up or down based on traffic volume forecasts, including:
[0143] Get the number of database service instances at the current moment, and the business capacity determined based on the number of database service instances at the current moment. Here, a database service instance refers to an instance used to provide database services. These instances together form a database cluster. In addition, each instance can carry a certain amount of business. The business capacity here can be the business capacity of one instance or the business capacity of multiple business instances.
[0144] When the difference between the detected service capacity and the predicted service capacity is greater than a first preset threshold, it is determined that the database needs to be expanded. The first preset threshold can be set according to actual needs, such as 0. That is, when the difference between the predicted service capacity and the service capacity is greater than 0, it means that the database needs to carry more services in the future. Therefore, the database needs to be expanded in advance to reduce cold start latency.
[0145] When the difference between the detected service load and the predicted service load is less than a second preset threshold, it is determined that the database needs to be scaled down. The second preset threshold can be set according to actual needs. It can be the same as or different from the first preset threshold. For example, when the second preset threshold is 0, that is, when the difference between the predicted service load and the service load is less than 0, it means that the database needs to reduce the service load in the future. Therefore, the database needs to be scaled down in advance to reduce the database service cost. Alternatively, the second preset threshold can be a value less than 0, that is, a buffer time or buffer value is given when the service load needs to be reduced to avoid the fluctuation of service load causing a decrease in the continuity of database service provision.
[0146] In some optional implementations, the initial scaling-up / scaling-down quantity of the database service instance is determined based on whether the database needs to be scaled up or down, including:
[0147] When a database is detected to require scaling up or down, the first scaling up or down number of database service instances is determined based on the absolute value of the difference between the service capacity and the predicted service capacity, as well as the service capacity of each database service instance. Specifically, the first scaling up or down number of database service instances is determined by dividing the absolute value of the difference by the service capacity of each database service instance, rounding the result to the nearest whole number, or by adding the service capacities of each database service instance together. If the sum is greater than the absolute value of the difference, the number of database service instances used for the addition operation is determined as the first scaling up or down number of database service instances.
[0148] In the above implementation, the expansion / shrinkage capacity of the database is determined based on the business volume prediction results, and the database is expanded or shrunken in advance based on the expansion / shrinkage capacity, thereby reducing the cold start latency of the database service and improving the reliability of the database service.
[0149] S3: Based on real-time parameter information from the database, determine the business load pressure of the database. The real-time parameter information is used to describe the database's operating status.
[0150] It should be noted that the real-time parameter information of the database can include the resource utilization of the database service instance, request response latency, and wait queue length. Resource utilization includes at least the Central Processing Unit (CPU) utilization and memory utilization. The calculation method for request response latency includes:
[0151] When responding to a database distribution request, record relevant information about the database distribution request. This information includes at least the request arrival time (RAT), request forwarding time (RFT), and request response time (RRT).
[0152] Based on the relevant information, the request response time (RTT) is calculated by summing up the above relevant information.
[0153] In some embodiments, this step involves assessing the load on the service within the elastic scaling strategy, as shown in Figure 3. This elastic scaling strategy is stored in the elastic scaling controller (DB Service AutoScaler). It periodically checks the database's operating status and, based on the detection results, senses the pressure on the database service to determine whether to scale. The result is then passed to the Serverless module, which creates / destroys database service instances to achieve load-aware scaling. When there are no database service requests for an extended period, it works in conjunction with the Serverless module to scale down the number of database service instances to a preset value. The Serverless module can define scaling policies (AutoScaler Policy) as needed, specifying parameters such as scaling threshold range, scaling cooldown time, and detection cycle for the database service. The definitions are as follows:
[0154] 1.apiVersion:v1
[0155] 2. kind:ConfigMap
[0156] 3. metadata:
[0157] 4. name:serverless-db-autoscalerpolicy
[0158] 5. namespace: kube-system
[0159] 6.data:
[0160] 7. detect-interval:30s #Detection interval
[0161] 8. cooldown-time: 30s # Cooldown time adjustment
[0162] 9. scaleup:
[0163] 10. CPU Usage: 80% #CPU Utilization
[0164] 11. mem-usage: 80% # Memory usage
[0165] 12. qps-delay:5000ms #qps delay
[0166] 13. scaledown:
[0167] 14. CPU Usage: 20% # CPU Utilization
[0168] 15. mem-usage: 20% # Memory usage
[0169] 16. qps-delay:50ms #qps delay
[0170] 17.scale-zero:true # Whether to scale to 0
[0171] 18. immutable: true
[0172] In some optional implementations, the characteristic feature is that determining the database's service load capacity based on real-time parameter information includes:
[0173] Obtain real-time parameter information and normalize it. The data normalization method is a common technique and will not be elaborated here.
[0174] Using the first objective fitting function, the normalized real-time parameter information is fitted to obtain the first objective fitting value. The first objective fitting function includes: A = aw1 + bw2 + cw3
[0175] Where A represents the first target fitted value, w1, w2, and w3 all represent weight coefficients, a represents the resource utilization rate of the database service instance, b represents the request response latency, and c represents the waiting queue length.
[0176] Based on the first target fitted value, the business load pressure of the database is determined, including:
[0177] Based on the first mapping relationship between the first target fitted value and the database's service carrying pressure, the service carrying pressure of the database corresponding to the first target fitted value is determined. The mapping relationship between the first target fitted value and the database's service carrying pressure can be obtained based on expert experience. That is, the service carrying pressure corresponding to each fitted value is determined by expert experience. The larger the fitted value, the greater the service carrying pressure.
[0178] In some alternative implementations, based on the database's service load capacity, it is determined whether the database needs to be scaled up or down, including:
[0179] When the database's service load is detected to be greater than a third preset threshold, it is determined that the database needs to be expanded.
[0180] When the database's service load is detected to be less than the fourth preset threshold, it is determined that the database needs to be scaled down.
[0181] The third preset threshold and the fourth preset value can be set according to actual needs.
[0182] In some optional implementations, in response to detecting that the database needs to be scaled up or down, a second scaling up or down quantity for the database service instance is calculated and determined, including:
[0183] Obtain the scaling thresholds corresponding to real-time parameter information. The real-time parameter information includes at least resource utilization and request response latency. Resource utilization includes at least CPU utilization and memory utilization. The scaling thresholds may include CPU scaling thresholds, memory scaling thresholds, and request response latency scaling thresholds. The above scaling thresholds can be set according to actual needs.
[0184] Using a second objective fitting function, a second objective fitting value is calculated and determined, and the second objective fitting value is defined as the second scaling up / down quantity of the database service instance. The second objective fitting function includes:
[0185] Where D(t) represents the fitted value of the second objective, w4, w5, and w6 are all weight coefficients, and cpu usage(t) Indicates central processing unit utilization, CPU usage(s) Mem represents the central processing unit scaling threshold. usage(t) Indicates memory usage, mem usage(s) Indicates the memory scaling threshold, qps delay(t) Queries per second (qps) represent request-response latency. delay(s) This represents the threshold for extending the response time, where t represents time.
[0186] In the above implementation, the service load of the database is determined by the real-time parameter information of the database, so as to determine the real-time scaling capacity of the database based on the service load. This enables elastic scaling of the database based on the real-time scaling capacity, thereby improving the reliability and continuity of the database service.
[0187] S4: Based on the business volume forecast results and / or the business capacity pressure of the database, determine the target number of database service instances to be scaled up or down. The database service instances are used to provide database services.
[0188] It should be noted that the scaling up / down quantities for the future time period corresponding to the current moment are predicted based on the business volume forecast results, so as to scale the database up or down in advance. The scaling up / down quantities for the current moment are determined based on the business load pressure of the database, so as to correct the database scaling up / down results. The database service instance is a database service instance deployed on the Kubernetes (also known as K8s, an open source system for automatically deploying, scaling up and managing containerized applications) container platform when providing server-insensitive computing database services. It runs in a container. Kubernetes is an open source container cluster management system from Google. Its main functions include: container-based application deployment, maintenance and rolling upgrades; load balancing and service discovery; cross-server and cross-region cluster scheduling; autoscaling; stateless services and stateful services. A container is a portable, lightweight operating system-level virtualization technology. It uses namespaces to isolate different software runtime environments and self-contains the software runtime environment through images, so that containers can be easily deployed anywhere. Because containers are small and start quickly, an application can be packaged within each container image. This one-to-one application image relationship has many advantages. Using containers eliminates the need to be bound to external infrastructure environments, as each application has no external dependencies and is independent of external infrastructure, perfectly solving the consistency problem from development to production. Containers are also more transparent than virtual machines, which facilitates monitoring and management, especially since the lifecycle of container processes is managed by the infrastructure, rather than hidden inside the container by the process manager. Finally, since each application is encapsulated in a container, managing container deployments is equivalent to managing application deployments. Furthermore, serverless computing (also known as serverless or function computing) is an emerging cloud computing programming paradigm that aims to help developers solve the problem of automatic management of virtual resources. Developers only need to complete the development of business logic code (i.e., functions), while the cloud platform is responsible for configuring the hardware and software resources required for function execution, automatic scaling, monitoring, etc. Currently, the industry generally defines serverless computing as Function as a Service (FaaS). The combination of Serverless computing (AS as a Service) and Backend as a Service (BaaS) greatly simplifies the development process of cloud applications compared to the traditional cloud computing model represented by IaaS. At the same time, the pay-as-you-go billing method also helps to reduce the cost of cloud usage for users.
[0189] In some optional implementations, determining the target scaling up / down quantity for the database service instance based on traffic volume forecasts and / or the database's service load capacity includes:
[0190] Obtain the first scaling up / down quantity determined based on the business volume forecast results, and the second scaling up / down quantity determined based on the business load pressure of the database;
[0191] In response to detecting that the current time is within the first target time period, the first scaling up or down quantity is determined to be the target scaling up or down quantity of the database service instances within the second target time period. The first target time period is the time period during which scaling up or down has been performed based on the business volume prediction results, and the second target time period is the future time period during which scaling up or down has not been performed.
[0192] Based on the second scaling factor, determine the target scaling factor for the database service instance at the current moment, including:
[0193] In response to the detection that expansion is needed at the current moment, the second expansion / shrinkage quantity is defined as the target expansion / shrinkage quantity of the database service instance at the current moment;
[0194] In response to the detection that scaling down is required at the current moment, the target value for scaling up or down is obtained within the third target time period, which is the time period preceding the time period during which scaling up or down has already occurred.
[0195] In response to the detection that the target value is less than the fifth preset threshold, the second scaling-up / scaling quantity is defined as the target scaling-up / scaling quantity of the database service instance at the current moment. When the target value is greater than or equal to the fifth preset threshold, scaling-up is temporarily suspended. The fifth preset threshold can be set according to actual needs. The target value is used to describe the volatility of the scaling-up / scaling capacity in the previous time period. The calculation method of the target value includes:
[0196] Obtain the number of expansions and contractions corresponding to multiple time nodes within the third target time period, and sort the number of expansions and contractions in order of time from oldest to newest, that is, the number of expansions and contractions at earlier times is listed later, and the number of expansions and contractions at later times is listed earlier. For example, if the time is 1 o'clock, 2 o'clock, and 3 o'clock, the corresponding number of expansions and contractions is x1, x2, and x3, then the sorting is x3, x2, and x1.
[0197] The target value is obtained by fitting the scaling up and down quantities corresponding to multiple time points using a third objective fitting function. The third objective fitting function includes: B = (x1ε1 + x2ε2 + ... + x n ε n ) / n
[0198] Where B represents the target value, and X = x1, x2, ..., x n Y represents the expansion / shrinkage quantity corresponding to the 1st, 2nd, ..., nth time points, where Y = ε1, ε2, ..., ε nThis represents the weighting coefficient corresponding to each expansion / shrinkage capacity, where n represents the number of time nodes. The larger n is, the larger the weighting coefficient is, meaning the closer it is to the current time, the greater its influence on the calculation result, i.e., the target value. This better reflects the fluctuation pattern of the expansion / shrinkage capacity at the current time, thus making the result of determining the target expansion / shrinkage capacity based on this target value more accurate.
[0199] In the above implementation, different target scaling-up and scaling-down quantities are selected according to different time periods. This ensures that the database can be scaled up or down in advance, and that the scaling-up and scaling-down results can be corrected according to real-time business volume after the advance scaling-up and scaling-down to meet real-time business needs. In addition, when scaling down is required at the current moment, the scaling-up and scaling-down volatility of the previous period is calculated. When the volatility is small, scaling down is performed; when the volatility is large, scaling down is not performed for the time being. This avoids the problem of server performance degradation caused by repeated scaling up and down of the database in a short period of time and reduces operating costs.
[0200] S5: Based on the target scaling size, scale the database to respond to database distribution requests.
[0201] It should be noted that the database distribution request, also known as a database request, represents a database connection request initiated on the application side. Based on this connection request, the corresponding database is matched and the matched database resources are distributed. This request is distributed through the Serverless DB Proxy module, which allocates database resources on demand. It provides a unified database service to business systems via NodePort or load balancer, allowing business systems to establish database connections. Business systems access the service using the HostIP (Internet Protocol):NodePort / ServiceName:namespace format, or they use the load balancer's IP address to establish a connection. This provides connection services to database service users and distributes requests according to load conditions, as shown in Figure 4. The module's responsibilities include: providing a unified database connection service for applications such as Application 1...Application 5; maintaining each database connection pool and the database service instances stored in the connection pools. Instance), manages the database connections of the application; distributes and forwards database requests to achieve load balancing among database service instances and ensures the consistency of client connections; records the number of database service requests and the response latency of each request, and stores them in a time-series database; service discovery after database service instances go online or offline.
[0202] In some optional implementations, scaling the database based on the target scaling size includes:
[0203] Check whether the current database is within the scaling-down cooldown cycle. The scaling-down cooldown cycle refers to the cooldown time set when the database is elastically scaling. This cooldown time is a configurable setting that can be set according to actual needs, such as setting the scaling-up cooldown cycle to 3 minutes and the scaling-down cooldown cycle to 5 minutes. Of course, it can also be set to other time ranges, which are not limited here.
[0204] If the database is not within the expansion / shrinkage cooldown period, it will be expanded or shrunken based on the target expansion / shrinkage amount. If it is within the expansion / shrinkage cooldown period, the expansion / shrinkage will be carried out after the cooldown period ends, or staff will manually adjust the scale based on their experience to determine whether expansion / shrinkage is possible.
[0205] In some alternative implementations, the method further includes, prior to responding to a database distribution request:
[0206] Obtain the identification information carried in the database distribution request. This identification information refers to the unique identifier of the application carried when the application initiates a data connection request. It can be used to determine what the available applications are in order to match the corresponding database.
[0207] Based on the identification information, the historical connection records between the database distribution request initiator and the target database service instance are determined. These historical connection records are stored in the database in the form of "Key-Value". The Key-Value database (KV database) is a type of database that stores data in key-value pairs and is connected to the database service proxy module.
[0208] When a historical connection record is detected and the target database service instance still exists, it means that the application has previously connected to the target database service instance. If the backend of the target database service instance still exists, the database distribution request will be forwarded to the target database service instance first.
[0209] In response to the detection that no historical connection record exists and / or the target database service instance does not exist, indicating that the application has never connected to the target database service instance, the database service instance to be used to forward the database distribution request is determined based on the business load of each database service instance. Optionally, if no historical connection record is found, the load pressure of the backend of each database service instance will be queried, and the current request will be preferentially connected to the database service instance with the lowest load pressure. The connection history will be recorded in the database in the form of "Key-Value" to facilitate subsequent connections.
[0210] In some embodiments, during the database request distribution process, the request arrival time (RAT), request forwarding time (RFT), and request response time (RRT) are recorded in detail to calculate the request response latency (RTT) based on the above data, thereby facilitating the determination of the service load.
[0211] In some embodiments, when the scaling controller triggers the database service instance to go online or offline according to the user-defined policy, the database service agent will complete the service discovery after the database service instance goes online or offline based on the Kubernetes platform service discovery mechanism. This ensures that when a new instance comes online, the database request service initiated by the application side can be distributed to the new instance. When some instances go offline, requests will no longer be distributed to the corresponding instance, and the corresponding information will be deleted from the database connection record.
[0212] In some alternative implementations, the method further includes:
[0213] If no database distribution request is detected within the preset time range, the number of database service instances will be reduced to the preset value. The preset time range and preset value can be set according to actual needs. The preset value is generally set to 0, which means that when there are no service requests for a period of time, the number of database service instances will be reduced to 0.
[0214] In the above implementation, the database is scaled up or down based on the target scaling quantity to meet real-time business needs, reduce operating costs, significantly improve the reliability and continuity of database services, accelerate the development and implementation of cloud-native technologies, and promote the rapid development of the industry.
[0215] The aforementioned database management method includes: determining historical business volume parameters based on historical time-series business data of the database, whereby the historical business volume parameters describe the development pattern of business volume; constructing a business volume prediction model based on the historical business volume parameters, and determining the business volume prediction result within a target time period based on the business volume prediction model; determining the business load pressure of the database based on real-time parameter information of the database, whereby the real-time parameter information describes the database operating status; determining the target scaling quantity corresponding to the database service instance based on the business volume prediction result and / or the business load pressure of the database, whereby the database service instance is used to provide database services; scaling the database based on the target scaling quantity to respond to database distribution requests. In order to reduce the response latency of requests, this application issues elastic scaling requests in advance based on the prediction results of the prediction model, and distributes requests according to the load situation through the database service proxy module, reducing cold start latency. In addition, the elastic scaling controller will trigger the Serverless module to quickly scale the database service instance according to the changes in database service pressure. When there is no business pressure, the database service instance is reduced to 0, reducing the service provision cost while meeting business performance requirements, thereby providing high-performance, highly available, server-insensitive database services at low cost.
[0216] It should be understood that although the steps in the flowcharts of Figures 2-4 are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in Figures 2-4 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0217] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in Figure 5. The computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile readable storage medium and internal memory. The non-volatile readable storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile readable storage medium. The network interface of the computer device is configured to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a database management method. The display screen of the computer device may be a liquid crystal display (LCD) or an electronic ink display. The input device of the computer device may be a touch layer covering the display screen, or buttons, a trackball, or a touchpad located on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.
[0218] Those skilled in the art will understand that the structure shown in Figure 5 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. For example, the computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0219] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:
[0220] S1: Based on historical time-series business data from the database, determine historical business volume parameters, which are used to describe the development pattern of business volume.
[0221] S2: Based on historical business volume parameters, construct a business volume prediction model, and based on the business volume prediction model, determine the business volume prediction results for the target time period;
[0222] S3: Based on real-time parameter information from the database, determine the business load pressure of the database. The real-time parameter information is used to describe the database's operating status.
[0223] S4: Based on the business volume forecast results and / or the business carrying pressure of the database, determine the target number of database service instances to be scaled up or down. The database service instances are used to provide database services.
[0224] S5: Based on the target scaling size, scale the database to respond to database distribution requests.
[0225] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0226] Obtain historical time-series business data from the database, and based on the historical time-series business data, determine the characteristic information of the historical time-series business data. The characteristic information includes at least time characteristics and business characteristics.
[0227] Construct historical business volume curves based on the feature information of historical time-series business data;
[0228] Based on the time series forecasting algorithm, the historical business volume curve is decomposed to obtain business volume parameters.
[0229] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0230] Based on historical business volume parameters, construct a business volume parameter model based on the objective function representation. The historical business volume parameters include at least trend components, periodic components, holiday parameters, and residual components.
[0231] Fit the business volume parameter model to construct a business volume prediction model.
[0232] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0233] Based on the first objective function and the trend component, a trend component model is constructed. The first objective function includes: T(t)=(1-α)T(t-1)
[0234] Where T(t) represents the trend component at time t, t represents time, α∈[0,1] represents the trend parameter, and T(t-1) represents the trend component at time t-1.
[0235] Based on the second objective function and the periodic component, a periodic component model is constructed. The second objective function includes: C(t)=(1-β)C(tk)
[0236] Where C(t) represents the periodic component at time t, k∈[0,1] represents the period length, β represents the period parameter, and C(tk) represents the trend component at time tk.
[0237] Based on the third objective function and holiday parameters, a holiday parameter model is constructed. The third objective function includes:
[0238] Where H(t) represents the holiday parameter at time t, and a∈[0,10000] represents the coefficient affected by the holiday;
[0239] Based on the fourth objective function and the residual components, a residual component model is constructed. The fourth objective function includes: R(t)=Y(t-1)-H(t-1)*T(t-1)*C(t-1)
[0240] Where R(t) represents the residual component at time t, Y(t-1) represents the predicted business volume at time t-1, H(t-1) represents the holiday parameter at time t-1, and C(t-1) represents the periodic component at time t-1.
[0241] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0242] The first, second, and third objective functions are multiplied together, and the fourth objective function is summed to construct a business volume prediction model, including: Y(t)=C(t)×H(t)×T(t)+R(t)
[0243] Where Y(t) represents the predicted traffic volume at time t.
[0244] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0245] Set the time period;
[0246] Based on the time period, determine the target time period corresponding to the current moment;
[0247] Based on the business volume prediction model, at least one business volume prediction value is obtained within the target time period to generate the business volume prediction result for the target time period.
[0248] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0249] Get the number of database service instances at the current moment, and the business capacity determined based on the number of database service instances at the current moment;
[0250] When the difference between the detected service capacity and the predicted service capacity is greater than a first preset threshold, it is determined that the database needs to be expanded.
[0251] When the difference between the detected service capacity and the predicted service capacity is less than a second preset threshold, it is determined that the database needs to be scaled down.
[0252] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0253] In response to the detection that the database needs to be scaled up or down, the first scaling up or down number of database service instances is determined based on the absolute value of the difference between the service capacity and the predicted service capacity, as well as the service capacity of each database service instance.
[0254] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0255] Obtain real-time parameter information and normalize it. The real-time parameter information includes at least one of the following: resource utilization rate of the database service instance, request response latency, and waiting queue length.
[0256] Using the first objective fitting function, the normalized real-time parameter information is fitted to obtain the first objective fitting value. The first objective fitting function includes: A = aw1 + bw2 + cw3
[0257] Where A represents the first target fitted value, w1, w2, and w3 all represent weight coefficients, a represents the resource utilization rate of the database service instance, b represents the request response latency, and c represents the waiting queue length.
[0258] Based on the fitted value of the first objective, the business load pressure of the database is determined.
[0259] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0260] Based on the first mapping relationship between the first target fitted value and the database's service load pressure, the service load pressure of the database corresponding to the first target fitted value is determined.
[0261] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0262] When the database's service load is detected to be greater than a third preset threshold, it is determined that the database needs to be expanded.
[0263] When the database's service load is detected to be less than the fourth preset threshold, it is determined that the database needs to be scaled down.
[0264] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0265] Obtain the scaling threshold corresponding to real-time parameter information. The real-time parameter information includes at least resource utilization and request response latency. Resource utilization includes at least CPU utilization and memory utilization.
[0266] Using a second objective fitting function, the second objective fitting value is calculated and determined, and the second objective fitting value is defined as the second scaling up / down quantity of the database service instance. The second objective fitting function includes:
[0267] Where D(t) represents the fitted value of the second objective, w4, w5, and w6 are all weight coefficients, and cpu usage(t) Indicates central processing unit utilization, CPU usage(s) Mem represents the central processing unit scaling threshold. usage(t) Indicates memory usage, mem usage(s) Indicates the memory scaling threshold, qps delay(t) Queries per second (qps) represent request-response latency. delay(s) This represents the threshold for extending the response time, where t represents time.
[0268] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0269] Obtain the first scaling up / down quantity determined based on the business volume forecast results, and the second scaling up / down quantity determined based on the business load pressure of the database;
[0270] In response to detecting that the current time is within the first target time period, the first scaling up or down quantity is determined to be the target scaling up or down quantity of the database service instances within the second target time period. The first target time period is the time period during which scaling up or down has been performed based on the business volume prediction results, and the second target time period is the future time period during which scaling up or down has not been performed.
[0271] Based on the second scaling-up / scaling quantity, determine the target scaling-up / scaling quantity of the database service instance at the current moment.
[0272] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0273] In response to the detection that expansion is needed at the current moment, the second expansion / shrinkage quantity is defined as the target expansion / shrinkage quantity of the database service instance at the current moment;
[0274] In response to the detection that scaling down is required at the current moment, the target value for scaling up or down is obtained within the third target time period, which is the time period preceding the time period during which scaling up or down has already occurred.
[0275] In response to the detection that the target value is less than the fifth preset threshold, the second scaling up / down quantity is defined as the target scaling up / down quantity of the database service instance at the current moment.
[0276] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0277] Obtain the number of expansions and contractions corresponding to multiple time nodes within the third target time period, and sort the number of expansions and contractions in order of time from oldest to newest.
[0278] The target value is obtained by fitting the scaling up and down quantities corresponding to multiple time points using a third objective fitting function. The third objective fitting function includes: B = (x1ε1 + x2ε2 + ... + x n ε n ) / n
[0279] Where B represents the target value, and X = x1, x2, ..., x n Y represents the expansion / shrinkage quantity corresponding to the 1st, 2nd, ..., nth time points, where Y = ε1, ε2, ..., ε n This represents the weighting coefficient corresponding to each expansion / contraction capacity. The larger n is, the larger the weighting coefficient is. n represents the number of time nodes.
[0280] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0281] Check if the current time database is in a scaling-down cooling cycle;
[0282] If the database is not in a scaling-down cooldown period, it will be scaled up or down based on the target scaling-down quantity.
[0283] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0284] Obtain the identification information carried in the database distribution request;
[0285] Based on the identification information, determine the historical connection records between the database distribution request initiator and the target database service instance;
[0286] When a historical connection record is detected and the target database service instance still exists, the database distribution request is forwarded to the target database service instance.
[0287] In response to the detection that no historical connection record exists and / or the target database service instance does not exist, the database service instance to be used for forwarding the database distribution request is determined based on the business carrying capacity of each database service instance.
[0288] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0289] Obtain relevant information about the database distribution request, including at least the request arrival time, request forwarding time, and request return time;
[0290] Based on relevant information, the request-response latency is calculated and determined.
[0291] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0292] If no database distribution request is detected within the preset time range, the number of database service instances will be reduced to the preset value.
[0293] In one embodiment, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, the computer program performing the following steps when executed by a processor:
[0294] S1: Based on historical time-series business data from the database, determine historical business volume parameters, which are used to describe the development pattern of business volume.
[0295] S2: Based on historical business volume parameters, construct a business volume prediction model, and based on the business volume prediction model, determine the business volume prediction results for the target time period;
[0296] S3: Based on real-time parameter information from the database, determine the business load pressure of the database. The real-time parameter information is used to describe the database's operating status.
[0297] S4: Based on the business volume forecast results and / or the business carrying pressure of the database, determine the target number of database service instances to be scaled up or down. The database service instances are used to provide database services.
[0298] S5: Based on the target scaling size, scale the database to respond to database distribution requests.
[0299] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0300] Obtain historical time-series business data from the database, and based on the historical time-series business data, determine the characteristic information of the historical time-series business data. The characteristic information includes at least time characteristics and business characteristics.
[0301] Construct historical business volume curves based on the feature information of historical time-series business data;
[0302] Based on the time series forecasting algorithm, the historical business volume curve is decomposed to obtain business volume parameters.
[0303] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0304] Based on historical business volume parameters, construct a business volume parameter model based on the objective function representation. The historical business volume parameters include at least trend components, periodic components, holiday parameters, and residual components.
[0305] Fit the business volume parameter model to construct a business volume prediction model.
[0306] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0307] Based on the first objective function and the trend component, a trend component model is constructed. The first objective function includes: T(t)=(1-ɑ)T(t-1)
[0308] Where T(t) represents the trend component at time t, t represents time, α∈[0,1] represents the trend parameter, and T(t-1) represents the trend component at time t-1.
[0309] Based on the second objective function and the periodic component, a periodic component model is constructed. The second objective function includes: C(t)=(1-β)C(tk)
[0310] Where C(t) represents the periodic component at time t, k∈[0,1] represents the period length, β represents the period parameter, and C(tk) represents the trend component at time tk.
[0311] Based on the third objective function and holiday parameters, a holiday parameter model is constructed. The third objective function includes:
[0312] Where H(t) represents the holiday parameter at time t, and a∈[0,10000] represents the coefficient affected by the holiday;
[0313] Based on the fourth objective function and the residual components, a residual component model is constructed. The fourth objective function includes: R(t)=Y(t-1)-H(t-1)*T(t-1)*C(t-1)
[0314] Where R(t) represents the residual component at time t, Y(t-1) represents the predicted business volume at time t-1, H(t-1) represents the holiday parameter at time t-1, and C(t-1) represents the periodic component at time t-1.
[0315] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0316] The first, second, and third objective functions are multiplied together, and the fourth objective function is summed to construct a business volume prediction model, including: Y(t)=C(t)×H(t)×T(t)+R(t)
[0317] Where Y(t) represents the predicted traffic volume at time t.
[0318] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0319] Set the time period;
[0320] Based on the time period, determine the target time period corresponding to the current moment;
[0321] Based on the business volume prediction model, at least one business volume prediction value is obtained within the target time period to generate the business volume prediction result for the target time period.
[0322] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0323] Get the number of database service instances at the current moment, and the business capacity determined based on the number of database service instances at the current moment;
[0324] When the difference between the detected service capacity and the predicted service capacity is greater than a first preset threshold, it is determined that the database needs to be expanded.
[0325] When the difference between the detected service capacity and the predicted service capacity is less than a second preset threshold, it is determined that the database needs to be scaled down.
[0326] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0327] In response to the detection that the database needs to be scaled up or down, the first scaling up or down number of database service instances is determined based on the absolute value of the difference between the service capacity and the predicted service capacity, as well as the service capacity of each database service instance.
[0328] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0329] Obtain real-time parameter information and normalize it. The real-time parameter information includes at least one of the following: resource utilization rate of the database service instance, request response latency, and waiting queue length.
[0330] Using the first objective fitting function, the normalized real-time parameter information is fitted to obtain the first objective fitting value. The first objective fitting function includes: A = aw1 + bw2 + cw3
[0331] Where A represents the first target fitted value, w1, w2, and w3 all represent weight coefficients, α represents the resource utilization rate of the database service instance, b represents the request response latency, and c represents the waiting queue length.
[0332] Based on the fitted value of the first objective, the business load pressure of the database is determined.
[0333] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0334] Based on the first mapping relationship between the first target fitted value and the database's service load pressure, the service load pressure of the database corresponding to the first target fitted value is determined.
[0335] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0336] When the database's service load is detected to be greater than a third preset threshold, it is determined that the database needs to be expanded.
[0337] When the database's service load is detected to be less than the fourth preset threshold, it is determined that the database needs to be scaled down.
[0338] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0339] Obtain the scaling threshold corresponding to real-time parameter information. The real-time parameter information includes at least resource utilization and request response latency. Resource utilization includes at least CPU utilization and memory utilization.
[0340] Using a second objective fitting function, the second objective fitting value is calculated and determined, and the second objective fitting value is defined as the second scaling up / down quantity of the database service instance. The second objective fitting function includes:
[0341] Where D(t) represents the fitted value of the second objective, w4, w5, and w6 are all weight coefficients, and cpu usage(t) Indicates central processing unit utilization, CPU usage(s) Mem represents the central processing unit scaling threshold. usage(t) Indicates memory usage, mem usage(s) Indicates the memory scaling threshold, qps delay(t) Queries per second (qps) represent request-response latency. delay(s) This represents the threshold for extending the response time, where t represents time.
[0342] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0343] Obtain the first scaling up / down quantity determined based on the business volume forecast results, and the second scaling up / down quantity determined based on the business load pressure of the database;
[0344] In response to detecting that the current time is within the first target time period, the first scaling up or down quantity is determined to be the target scaling up or down quantity of the database service instances within the second target time period. The first target time period is the time period during which scaling up or down has been performed based on the business volume prediction results, and the second target time period is the future time period during which scaling up or down has not been performed.
[0345] Based on the second scaling-up / scaling quantity, determine the target scaling-up / scaling quantity of the database service instance at the current moment.
[0346] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0347] In response to the detection that expansion is needed at the current moment, the second expansion / shrinkage quantity is defined as the target expansion / shrinkage quantity of the database service instance at the current moment;
[0348] In response to the detection that scaling down is required at the current moment, the target value for scaling up or down is obtained within the third target time period, which is the time period preceding the time period during which scaling up or down has already occurred.
[0349] In response to the detection that the target value is less than the fifth preset threshold, the second scaling up / down quantity is defined as the target scaling up / down quantity of the database service instance at the current moment.
[0350] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0351] Obtain the number of expansions and contractions corresponding to multiple time nodes within the third target time period, and sort the number of expansions and contractions in order of time from oldest to newest.
[0352] The target value is obtained by fitting the scaling up and down quantities corresponding to multiple time points using a third objective fitting function. The third objective fitting function includes: B = (x1ε1 + x2ε2 + ... + x n ε n ) / n
[0353] Where B represents the target value, and X = x1, x2, ..., x n Y represents the expansion / shrinkage quantity corresponding to the 1st, 2nd, ..., nth time points, where Y = ε1, ε2, ..., ε n This represents the weighting coefficient corresponding to each expansion / contraction capacity. The larger n is, the larger the weighting coefficient is. n represents the number of time nodes.
[0354] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0355] Check if the current time database is in a scaling-down cooling cycle;
[0356] If the database is not in a scaling-down cooldown period, it will be scaled up or down based on the target scaling-down quantity.
[0357] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0358] Obtain the identification information carried in the database distribution request;
[0359] Based on the identification information, determine the historical connection records between the database distribution request initiator and the target database service instance;
[0360] When a historical connection record is detected and the target database service instance still exists, the database distribution request is forwarded to the target database service instance.
[0361] In response to the detection that no historical connection record exists and / or the target database service instance does not exist, the database service instance to be used for forwarding the database distribution request is determined based on the business carrying capacity of each database service instance.
[0362] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0363] Obtain relevant information about the database distribution request, including at least the request arrival time, request forwarding time, and request return time;
[0364] Based on relevant information, the request-response latency is calculated and determined.
[0365] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0366] If no database distribution request is detected within the preset time range, the number of database service instances will be reduced to the preset value.
[0367] In one embodiment, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, performs the following steps:
[0368] S1: Based on historical time-series business data from the database, determine historical business volume parameters, which are used to describe the development pattern of business volume.
[0369] S2: Based on historical business volume parameters, construct a business volume prediction model, and based on the business volume prediction model, determine the business volume prediction results for the target time period;
[0370] S3: Based on real-time parameter information from the database, determine the business load pressure of the database. The real-time parameter information is used to describe the database's operating status.
[0371] S4: Based on the business volume forecast results and / or the business carrying pressure of the database, determine the target number of database service instances to be scaled up or down. The database service instances are used to provide database services.
[0372] S5: Based on the target scaling size, scale the database to respond to database distribution requests.
[0373] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0374] Obtain historical time-series business data from the database, and based on the historical time-series business data, determine the characteristic information of the historical time-series business data. The characteristic information includes at least time characteristics and business characteristics.
[0375] Construct historical business volume curves based on the feature information of historical time-series business data;
[0376] Based on the time series forecasting algorithm, the historical business volume curve is decomposed to obtain business volume parameters.
[0377] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0378] Based on historical business volume parameters, construct a business volume parameter model based on the objective function representation. The historical business volume parameters include at least trend components, periodic components, holiday parameters, and residual components.
[0379] Fit the business volume parameter model to construct a business volume prediction model.
[0380] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0381] Based on the first objective function and the trend component, a trend component model is constructed. The first objective function includes: T(t)=(1-ɑ)T(t-1)
[0382] Where T(t) represents the trend component at time t, t represents time, α∈[0,1] represents the trend parameter, and T(t-1) represents the trend component at time t-1.
[0383] Based on the second objective function and the periodic component, a periodic component model is constructed. The second objective function includes: C(t)=(1-β)C(tk)
[0384] Where C(t) represents the periodic component at time t, k∈[0,1] represents the period length, β represents the period parameter, and C(tk) represents the trend component at time tk.
[0385] Based on the third objective function and holiday parameters, a holiday parameter model is constructed. The third objective function includes:
[0386] Where H(t) represents the holiday parameter at time t, and a∈[0,10000] represents the coefficient affected by the holiday;
[0387] Based on the fourth objective function and the residual components, a residual component model is constructed. The fourth objective function includes: R(t)=Y(t-1)-H(t-1)*T(t-1)*C(t-1)
[0388] Where R(t) represents the residual component at time t, Y(t-1) represents the predicted business volume at time t-1, H(t-1) represents the holiday parameter at time t-1, and C(t-1) represents the periodic component at time t-1.
[0389] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0390] The first, second, and third objective functions are multiplied together, and the fourth objective function is summed to construct a business volume prediction model, including: Y(t)=C(t)×H(t)×T(t)+R(t)
[0391] Where Y(t) represents the predicted traffic volume at time t.
[0392] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0393] Set the time period;
[0394] Based on the time period, determine the target time period corresponding to the current moment;
[0395] Based on the business volume prediction model, at least one business volume prediction value is obtained within the target time period to generate the business volume prediction result for the target time period.
[0396] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0397] Get the number of database service instances at the current moment, and the business capacity determined based on the number of database service instances at the current moment;
[0398] When the difference between the detected service capacity and the predicted service capacity is greater than a first preset threshold, it is determined that the database needs to be expanded.
[0399] When the difference between the detected service capacity and the predicted service capacity is less than a second preset threshold, it is determined that the database needs to be scaled down.
[0400] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0401] In response to the detection that the database needs to be scaled up or down, the first scaling up or down number of database service instances is determined based on the absolute value of the difference between the service capacity and the predicted service capacity, as well as the service capacity of each database service instance.
[0402] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0403] Obtain real-time parameter information and normalize it. The real-time parameter information includes at least one of the following: resource utilization rate of the database service instance, request response latency, and waiting queue length.
[0404] Using the first objective fitting function, the normalized real-time parameter information is fitted to obtain the first objective fitting value. The first objective fitting function includes: A = aw1 + bw2 + cw3
[0405] Where A represents the first target fitted value, w1, w2, and w3 all represent weight coefficients, a represents the resource utilization rate of the database service instance, b represents the request response latency, and c represents the waiting queue length.
[0406] Based on the fitted value of the first objective, the business load pressure of the database is determined.
[0407] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0408] Based on the first mapping relationship between the first target fitted value and the database's service load pressure, the service load pressure of the database corresponding to the first target fitted value is determined.
[0409] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0410] When the database's service load is detected to be greater than a third preset threshold, it is determined that the database needs to be expanded.
[0411] When the database's service load is detected to be less than the fourth preset threshold, it is determined that the database needs to be scaled down.
[0412] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0413] Obtain the scaling threshold corresponding to real-time parameter information. The real-time parameter information includes at least resource utilization and request response latency. Resource utilization includes at least CPU utilization and memory utilization.
[0414] Using a second objective fitting function, the second objective fitting value is calculated and determined, and the second objective fitting value is defined as the second scaling up / down quantity of the database service instance. The second objective fitting function includes:
[0415] Where D(t) represents the fitted value of the second objective, w4, w5, and w6 are all weight coefficients, and cpu usage(t) Indicates central processing unit utilization, CPU usage(s) Mem represents the central processing unit scaling threshold. usage(t) Indicates memory usage, memusage(s) Indicates the memory scaling threshold, qps delay(t) Queries per second (qps) represent request-response latency. delay(s) This represents the threshold for extending the response time, where t represents time.
[0416] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0417] Obtain the first scaling up / down quantity determined based on the business volume forecast results, and the second scaling up / down quantity determined based on the business load pressure of the database;
[0418] In response to detecting that the current time is within the first target time period, the first scaling up or down quantity is determined to be the target scaling up or down quantity of the database service instances within the second target time period. The first target time period is the time period during which scaling up or down has been performed based on the business volume prediction results, and the second target time period is the future time period during which scaling up or down has not been performed.
[0419] Based on the second scaling-up / scaling quantity, determine the target scaling-up / scaling quantity of the database service instance at the current moment.
[0420] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0421] In response to the detection that expansion is needed at the current moment, the second expansion / shrinkage quantity is defined as the target expansion / shrinkage quantity of the database service instance at the current moment;
[0422] In response to the detection that scaling down is required at the current moment, the target value for scaling up or down is obtained within the third target time period, which is the time period preceding the time period during which scaling up or down has already occurred.
[0423] In response to the detection that the target value is less than the fifth preset threshold, the second scaling up / down quantity is defined as the target scaling up / down quantity of the database service instance at the current moment.
[0424] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0425] Obtain the number of expansions and contractions corresponding to multiple time nodes within the third target time period, and sort the number of expansions and contractions in order of time from oldest to newest.
[0426] The target value is obtained by fitting the scaling up and down quantities corresponding to multiple time points using a third objective fitting function. The third objective fitting function includes: B = (x1ε1 + x2ε2 + ... + x n ε n ) / n
[0427] Where B represents the target value, and X = x1, x2, ..., xn Y represents the expansion / shrinkage quantity corresponding to the 1st, 2nd, ..., nth time points, where Y = ε1, ε2, ..., ε n This represents the weighting coefficient corresponding to each expansion / contraction capacity. The larger n is, the larger the weighting coefficient is. n represents the number of time nodes.
[0428] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0429] Check if the current time database is in a scaling-down cooling cycle;
[0430] If the database is not in a scaling-down cooldown period, it will be scaled up or down based on the target scaling-down quantity.
[0431] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0432] Obtain the identification information carried in the database distribution request;
[0433] Based on the identification information, determine the historical connection records between the database distribution request initiator and the target database service instance;
[0434] When a historical connection record is detected and the target database service instance still exists, the database distribution request is forwarded to the target database service instance.
[0435] In response to the detection that no historical connection record exists and / or the target database service instance does not exist, the database service instance to be used for forwarding the database distribution request is determined based on the business carrying capacity of each database service instance.
[0436] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0437] Obtain relevant information about the database distribution request, including at least the request arrival time, request forwarding time, and request return time;
[0438] Based on relevant information, the request-response latency is calculated and determined.
[0439] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0440] If no database distribution request is detected within the preset time range, the number of database service instances will be reduced to the preset value.
[0441] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous Dynamic RAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchlink Dynamic RAM (SLDRAM), Rambus Direct RAM (RDRAM), Direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), etc.
[0442] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0443] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are quite detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A database management method, characterized in that, The method includes: Based on historical time-series business data from the database, historical business volume parameters are determined, which are used to describe the development pattern of business volume. Based on the historical traffic volume parameters, a traffic volume prediction model is constructed, and based on the traffic volume prediction model, the traffic volume prediction result for the target time period is determined. Based on real-time parameter information of the database, the business load pressure of the database is determined, and the real-time parameter information is used to describe the database operating status; Based on the traffic volume forecast results and / or the business carrying pressure of the database, determine the target scaling up or down number of database service instances, which are used to provide database services. Based on the target scaling size, the database is scaled up or down in response to database distribution requests.
2. The database management method according to claim 1, characterized in that, Based on historical time-series business data from the database, historical business volume parameters are determined as follows: Obtain historical time-series business data from the database, and based on the historical time-series business data, determine the characteristic information of the historical time-series business data, wherein the characteristic information includes at least time characteristics and business characteristics; Based on the feature information of the historical time-series business data, a historical business volume curve is constructed; Based on the time series prediction algorithm, the historical traffic volume curve is decomposed to obtain traffic volume parameters.
3. The database management method according to claim 1, characterized in that, Based on the historical traffic volume parameters, the traffic volume prediction model is constructed as follows: Based on the historical traffic volume parameters, a traffic volume parameter model based on an objective function is constructed; The business volume parameter model is fitted to construct the business volume prediction model.
4. The database management method according to claim 3, characterized in that, The historical business volume parameters include at least trend components, periodic components, holiday parameters, and residual components. Based on these historical business volume parameters, constructing a business volume parameter model based on an objective function includes: Based on the first objective function and the trend components, a trend component model is constructed. The first objective function includes: T(t)=(1-α)T(t-1) Where T(t) represents the trend component at time t, t represents time, ɑ∈[0,1] represents the trend parameter, and T(t-1) represents the trend component at time t-1. Based on the second objective function and the periodic component, a periodic component model is constructed. The second objective function includes: C(t)=(1-β)C(tk) Where C(t) represents the periodic component at time t, k∈[0,1] represents the period length, β represents the period parameter, and C(tk) represents the trend component at time tk. Based on the third objective function and the holiday parameters, a holiday parameter model is constructed. The third objective function includes: Where H(t) represents the holiday parameter at time t, and a∈[0,10000] represents the coefficient affected by the holiday; Based on the fourth objective function and the residual components, a residual component model is constructed. The fourth objective function includes: R(t)=Y(t-1)-H(t-1)*T(t-1)*C(t-1) Where R(t) represents the residual component at time t, Y(t-1) represents the predicted business volume at time t-1, H(t-1) represents the holiday parameter at time t-1, and C(t-1) represents the periodic component at time t-1.
5. The database management method according to claim 4, characterized in that, Fitting the traffic volume parameter model to construct the traffic volume prediction model includes: The business volume prediction model is constructed by multiplying the first objective function, the second objective function, and the third objective function, and then summing them with the fourth objective function. This includes: Y(t)=C(t)×H(t)×T(t)+R(t) Where Y(t) represents the predicted traffic volume at time t.
6. The database management method according to claim 1, characterized in that, Based on the aforementioned business volume forecasting model, the business volume forecast results for the target time period are determined as follows: Set the time period; Based on the time period, determine the target time period corresponding to the current moment; Based on the traffic volume prediction model, at least one traffic volume prediction value is obtained within the target time period to generate the traffic volume prediction result for the target time period.
7. The database management method according to claim 5 or 6, characterized in that, The method further includes determining whether the database needs to be scaled up or down based on the traffic volume forecast results, including: Obtain the number of database service instances at the current moment, and the service capacity determined based on the number of database service instances at the current moment; When the difference between the detected service carrying capacity and the predicted service volume is greater than a first preset threshold, it is determined that the database needs to be expanded. When the difference between the detected service carrying capacity and the predicted service volume is less than a second preset threshold, it is determined that the database needs to be scaled down.
8. The database management method according to claim 7, characterized in that, Based on the determination of whether the database needs to be scaled up or down, the initial scaling up / down quantity for the database service instance is determined as follows: In response to the detection that the database needs to be scaled up or down, the first scaling up or down number of the database service instances is determined based on the absolute value of the difference between the service capacity and the predicted service capacity, as well as the service capacity of each database service instance.
9. The database management method according to claim 1, characterized in that, Based on real-time parameter information from the database, the business load capacity of the database is determined, including: Obtain real-time parameter information and normalize the real-time parameter information, which includes at least one of the following: resource utilization rate of the database service instance, request response latency, and waiting queue length; Using a first objective fitting function, the normalized real-time parameter information is fitted to obtain a first objective fitting value. The first objective fitting function includes: A = aw1 + bw2 + cw3 Where A represents the first target fitted value, w1, w2, and w3 all represent weight coefficients, a represents the resource utilization rate of the database service instance, b represents the request response latency, and c represents the waiting queue length. Based on the first target fitted value, the business load pressure of the database is determined.
10. The database management method according to claim 9, characterized in that, Based on the first target fitted value, the business load capacity of the database is determined as follows: Based on the first mapping relationship between the first target fitted value and the database's service carrying pressure, the service carrying pressure of the database corresponding to the first target fitted value is determined.
11. The database management method according to claim 9 or 10, characterized in that, Based on the service load of the database, determine whether the database needs to be scaled up or down, including: When the service load of the database is detected to be greater than a third preset threshold, it is determined that the database needs to be expanded. When the service load of the database is detected to be less than a fourth preset threshold, it is determined that the database needs to be scaled down.
12. The database management method according to claim 11, characterized in that, In response to detecting that the database needs to be scaled up or down, a second scaling up / down quantity for the database service instance is calculated and determined, including: Obtain the scaling threshold corresponding to real-time parameter information, wherein the real-time parameter information includes at least resource utilization and request response latency, and the resource utilization includes at least CPU utilization and memory utilization. Using the second objective fitting function, the second objective fitting value is calculated and determined, and the second objective fitting value is defined as the second scaling up / down number of the database service instance.
13. The database management method according to claim 12, characterized in that, The second objective fitting function includes: Where D(t) represents the fitted value of the second objective, w4, w5, and w6 are all weight coefficients, and cpu usage (t) represents the CPU utilization rate. usage (s) represents the central processing unit scaling threshold, mem usage (t) represents memory utilization, mem usage (s) represents the memory scaling threshold, qps delay (t) represents the request-response latency, in qps. delay (s) represents the scaling threshold when the request is responded to, and t represents the time.
14. The database management method according to claim 1, characterized in that, Based on the traffic volume forecast results and / or the traffic load pressure of the database, the target scaling up / down quantity for the database service instance is determined as follows: Obtain the first scaling-up / scaling-down quantity determined based on the traffic volume prediction results, and the second scaling-up / scaling-down quantity determined based on the service load pressure of the database; In response to detecting that the current time is within a first target time period, the first scaling up or down quantity is determined to be the target scaling up or down quantity of the database service instances within a second target time period. The first target time period is the time period during which scaling up or down has been performed based on the business volume prediction results, and the second target time period is a future time period during which scaling up or down has not been performed. Based on the second scaling-up / scaling quantity, determine the target scaling-up / scaling quantity of the database service instance at the current moment.
15. The database management method according to claim 14, characterized in that, Based on the second scaling-up / scaling quantity, the target scaling-up / scaling quantity for the database service instance at the current moment is determined as follows: In response to the detection that expansion is needed at the current moment, the second expansion / shrinkage quantity is defined as the target expansion / shrinkage quantity of the database service instance at the current moment; In response to the detection that scaling down is required at the current moment, the target value for scaling up or down is obtained within a third target time period, wherein the third target time period is the time period preceding the time period in which scaling up or down has already occurred. In response to detecting that the target value is less than the fifth preset threshold, the second scaling up / down quantity is defined as the target scaling up / down quantity of the database service instance at the current moment.
16. The database management method according to claim 15, characterized in that, The method for calculating the target value includes: Obtain the number of expansions and contractions corresponding to multiple time nodes within the third target time period, and sort the number of expansions and contractions in order of time from oldest to newest. The target value is obtained by fitting the scaling up and down quantities corresponding to the plurality of time points using a third objective fitting function, wherein the third objective fitting function includes: B=(x1ε1+x2ε2+...+x n e n ) / n Where B represents the target value, and X = x1, x2, ..., x n Y represents the expansion / shrinkage quantity corresponding to the 1st, 2nd, ..., nth time points, where Y = ε1, ε2, ..., ε n This represents the weighting coefficient corresponding to each expansion / contraction capacity. The larger n is, the larger the weighting coefficient is. n represents the number of time nodes.
17. The database management method according to claim 1, characterized in that, Based on the target scaling size, scaling the database includes: Check if the current time database is in a scaling-down cooling cycle; If the database is not in a scaling-down cooling-off period, it will be scaled up or down based on the target scaling-down quantity.
18. The database management method according to claim 1, characterized in that, Before responding to the database distribution request, the method further includes: Obtain the identification information carried in the database distribution request; Based on the identification information, the historical connection records between the database distribution request initiator and the target database service instance are determined; When a historical connection record is detected and the target database service instance still exists, the database distribution request is forwarded to the target database service instance; In response to the detection that no historical connection record exists and / or the target database service instance does not exist, the database service instance to be used for forwarding the database distribution request is determined based on the business carrying capacity of each database service instance.
19. The database management method according to claim 1, characterized in that, When responding to a database distribution request, the method further includes: Obtain relevant information about the database distribution request, including at least the request arrival time, request forwarding time, and request return time; Based on the aforementioned information, the request-response latency is calculated and determined.
20. The database management method according to claim 1, characterized in that, The method further includes: If no database distribution request is detected within the preset time range, the number of database service instances will be reduced to a preset value.
21. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 20.
22. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 20.
23. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 20.
Citation Information
Patent Citations
Capacity expansion and shrinkage processing method and device for database, computer equipment and readable medium
CN110457287A
Internet of vehicles platform capacity expansion and contraction method and system, and storage medium
CN117221125A
Dynamic capacity expansion and contraction method and device for satellite edge computing service and storage medium
CN117573339A
Elastic capacity expansion and contraction method based on business and result guidance
CN118069291A
Database management method, computer equipment, storage medium and program product
CN118245471A
Cited By
Priority-based database Autovacuum process scheduling method, equipment, and media
CN122412111A