Database resource scaling method and device, equipment, medium and product
By predicting the future utilization rate of database resources and automatically adjusting it, the problem of unbalanced utilization rate of database resources in financial business scenarios is solved, timely expansion or reduction of resources is achieved, and business efficiency and resource utilization are improved.
Patent Information
- Application Number
- CN202411142092.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-29
AI Technical Summary
In financial business scenarios, database resources show unusual patterns in transaction volume and time periods, resulting in uneven database resource utilization, resulting in untimely expansion or shrinkage, affecting business or wasting resources.
By obtaining the usage rate of container target resources in multiple historical cycles, predicting the usage rate of future cycles, and automatically expanding or shrinking according to preset thresholds, ensuring that resources increase or decrease in time during peak business periods.
It realizes timely expansion or reduction of database resources, improves business efficiency and resource utilization, and reduces computing complexity and manual intervention costs.
Smart Images

Figure CN119046359A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to a method, device, equipment, medium and product for scaling database resources. Background Art
[0002] In financial business scenarios, transaction volume and time period are closely related, and even in some financial business scenarios, transaction volume and time period show very regular phenomena. When transaction volume and time period show very regular phenomena, database resource usage rate also shows regular phenomena with time.
[0003] However, in order to meet the database resource requirements during the peak business hours, the database is usually provided with the maximum configuration, and the database resources will not automatically increase or decrease over time or with changes in business volume. If there is a change in database resource requirements, the increase or decrease of database resources can be achieved through manual operations.
[0004] Manual intervention requires the formulation of plans, exhausts human resource costs, and may result in untimely expansion or reduction of capacity, affecting business or wasting resources. Summary of the invention
[0005] The present application provides a database resource scaling method, apparatus, device, medium and product to solve the problem of untimely expansion or reduction of capacity.
[0006] In a first aspect, the present application provides a method for scaling database resources, the method comprising:
[0007] Obtaining usage rates of target resources of the container in multiple historical periods, and predicting usage rates of the target resources in multiple future periods based on the usage rates of the target resources in multiple historical periods;
[0008] Among the utilization rates of the future multiple cycles, if the utilization rates of consecutive preset cycles are higher than a first threshold, the target resource is expanded; if the utilization rates of consecutive preset cycles are lower than a second threshold, the target resource is reduced.
[0009] In some embodiments, obtaining the usage rate of the target resource of the container in multiple historical periods specifically includes:
[0010] According to the total usage time of the target resource in each of the historical periods, obtaining the target resource being run in each of the historical periods;
[0011] The utilization rate of the target resource in each historical period is calculated according to the total amount of the target resource and the target resource being operated in each historical period.
[0012] In some embodiments, the method for obtaining the total amount of the target resource includes:
[0013] Obtaining the cycle and allowed running time of the target resource;
[0014] If the allowed running time of the target resource is greater than zero and the period of the target resource is not equal to zero, calculating the ratio between the allowed running time of the target resource and the period of the target resource to obtain the total amount of the target resource;
[0015] If the allowed running time of the target resource is less than or equal to zero, determining the total amount of the target resource as the total amount of available target resources;
[0016] Alternatively, when the setting information of the target resource includes the total amount of the target resource, the total amount of the target resource is acquired from the setting information of the target resource.
[0017] In some embodiments, predicting the usage rate of the target resource in multiple future periods based on the usage rate of the target resource in multiple historical periods specifically includes:
[0018] Calculating the average utilization rate of the target resource in multiple historical periods according to the utilization rate of the target resource in multiple historical periods as the predicted level value of the first future period;
[0019] Based on the smoothing coefficient, the utilization rate of the target resource in the Nth historical period, the predicted level value of the target resource in the N-1th period in the future, and the predicted trend value of the target resource in the N-1th period in the future, the predicted level value of the target resource in the N-1th period in the future is calculated; N is an integer greater than 1;
[0020] Calculate the predicted trend value of the target resource in the Nth period in the future based on the trend smoothing coefficient, the predicted level value of the target resource in the Nth period in the future, the predicted level value of the target resource in the N-1th period in the future, and the predicted trend value of the target resource in the N-1th period in the future;
[0021] The utilization rate of the target resource in the next N cycles is calculated according to the predicted level value of the target resource in the next N cycles and the predicted trend value of the target resource in the next N cycles.
[0022] In some embodiments, the plurality of historical periods include periods across seasons;
[0023] The calculating of the predicted level value of the target resource in the future N-th period based on the smoothing coefficient, the utilization rate of the target resource in the N-th historical period, the predicted level value of the target resource in the future N-1-th period, and the predicted trend value of the target resource in the future N-1-th period specifically includes:
[0024] Based on the smoothing coefficient, the utilization rate of the target resource in the Nth historical period, the predicted seasonal value of the target resource in the future Nmth period, the predicted level value of the target resource in the future N-1th period, and the predicted trend value of the target resource in the future N-1th period, the predicted level value of the target resource in the future N-1th period is calculated.
[0025] In some embodiments, calculating the average usage rate of the target resource in multiple historical periods according to the usage rate of the target resource in multiple historical periods specifically includes:
[0026] Checking whether the number of utilization rates of the target resource in multiple historical periods reaches a preset number;
[0027] If so, the average utilization rate of the target resource in the multiple historical periods is calculated according to the utilization rate of the target resource in the multiple historical periods.
[0028] In some embodiments, if the usage rate of consecutive preset periods is higher than a first threshold, the target resource is expanded, specifically including:
[0029] Determine whether among the usage rates of the future multiple periods, there are consecutive preset periods with usage rates higher than a first threshold;
[0030] If so, calculating a first percentage of the usage rate of the consecutive preset periods exceeding the first threshold;
[0031] Calculating an expansion capacity of the target resource according to the first percentage and the total amount of the target resource;
[0032] The target resource is expanded according to the expansion capacity of the target resource.
[0033] In some embodiments, the expanding the capacity of the target resource according to the expansion capacity of the target resource specifically includes:
[0034] Determine whether the number of expansion times within a preset time has reached a preset number of expansion times;
[0035] If not, the target resource is expanded according to the expansion capacity and the cycle of the target resource, or the number of cores matching the expansion capacity is increased for the target resource.
[0036] In some embodiments, if the usage rate of consecutive preset periods is lower than a second threshold, scaling down the target resource specifically includes:
[0037] Determine whether there are consecutive preset periods of usage rates lower than a second threshold value among the usage rates of the future multiple periods;
[0038] If so, calculating a second percentage of the portion of the continuous preset period whose usage rate is lower than the second threshold;
[0039] Calculating a reduced capacity of the target resource according to the second percentage and the total amount of the target resource;
[0040] The target resource is reduced in capacity according to the reduced capacity of the target resource.
[0041] In some embodiments, the reducing the capacity of the target resource according to the reduced capacity of the target resource specifically includes:
[0042] Determine whether the number of shrinkages within a preset time has reached a preset number of shrinkages;
[0043] If not, the target resource is scaled down according to the scaled down capacity and the period of the target resource, or the number of cores for the target resource is reduced to match the scaled down capacity.
[0044] In a second aspect, the present application provides a database resource expansion device, the device comprising:
[0045] A first processing module is used to obtain the usage rate of the target resource of the container in multiple historical periods, and predict the usage rate of the target resource in multiple future periods according to the usage rate of the target resource in multiple historical periods;
[0046] The second processing module is used to expand the target resource if the usage rate of the consecutive preset cycles is higher than the first threshold value among the usage rates of the multiple future cycles, and to reduce the target resource if the usage rate of the consecutive preset cycles is lower than the second threshold value.
[0047] In a third aspect, the present application provides an electronic device, including: a memory and a processor;
[0048] The memory is used to store instructions; the processor is used to call the instructions in the memory to execute the method in the first aspect and any possible design of the first aspect.
[0049] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer instructions. When at least one processor of an electronic device executes the computer instructions, the electronic device executes the method in the first aspect and any possible design of the first aspect.
[0050] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. When at least one processor of an electronic device executes the computer instructions, the electronic device executes the method in the first aspect and any possible design of the first aspect.
[0051] The database resource scaling method, device, equipment, medium and product provided in the present application obtain the usage rate of the target resource of the container in multiple historical periods, predict the usage rate of the target resource in multiple future periods based on the usage rate of the target resource in multiple historical periods, and if the usage rate in consecutive preset periods exceeds a first threshold, the target resource is expanded; if the usage rate in consecutive preset periods is lower than a second threshold, the target resource is reduced, thereby being able to expand or reduce the target resource in a timely manner, improving business efficiency and resource utilization, and having low computational complexity and higher efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0053] Figure 1 A flowchart of a database resource scaling method provided in an embodiment of the present application;
[0054] Figure 2 A flowchart of a database resource scaling method provided by another embodiment of the present application;
[0055] Figure 3 A flowchart of a database resource scaling method provided in yet another embodiment of the present application;
[0056] Figure 4 A flowchart of a database resource scaling method provided in yet another embodiment of the present application;
[0057] Figure 5 A flowchart of a database resource scaling method provided in yet another embodiment of the present application;
[0058] Figure 6 A schematic diagram of the structure of a database resource expansion device provided in an embodiment of the present application;
[0059] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0061] As described in the background art, database resources have changing demands. Manual operations to increase or decrease data resources may result in untimely expansion or reduction, which may affect business or waste resources.
[0062] In some embodiments, whether capacity expansion or contraction is required is determined based on current resource utilization. Capacity expansion or contraction may be delayed and cannot meet scenarios with high real-time requirements.
[0063] To this end, the present application proposes a database resource scaling method, which predicts the usage rate of the target resource in the future period based on the usage rate of the target resource of the container in the historical period, and expands the target resource when the usage rate of the consecutive preset periods exceeds a first threshold, and shrinks the target resource when the usage rate of the consecutive preset periods is lower than a second threshold, thereby enabling the target resource to be expanded or shrunk in a timely manner, improving business efficiency and resource utilization, and having low computational complexity and higher efficiency.
[0064] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0065] Figure 1 FIG. 1 is a flow chart showing a method for scaling database resources provided by an embodiment of the present application. Figure 1 As shown, with the electronic device as the execution subject, the method of this embodiment may include the following steps:
[0066] S101. Obtain usage rates of target resources of a container in multiple historical periods, and predict usage rates of the target resources in multiple future periods based on the usage rates of the target resources in multiple historical periods.
[0067] In actual applications, the database cloud platform is a new type of cloud computing service that can provide high-performance, high-reliability, high-security, and scalable database hosting services. The platform can provide various types of database services, such as relational database MySQL, non-relational database Redis, distributed database Tidb, etc. It can also provide backup and recovery, monitoring, rapid expansion, data transmission and other services.
[0068] Database cloud platforms usually deploy different business instances on a single server to achieve hybrid deployment and maximize resource utilization. At the same time, in order to reduce the mutual impact of different instances on the same server, resource isolation is usually required. Resource isolation methods include: physical resource isolation mechanism Cgroups (ControlGroups) provided by the Linux kernel; kernel-based virtual machine (KVM) technology; and kernel container (Docker) technology that depends on the operating system.
[0069] The database resource scaling method provided in the embodiment of the present application can be implemented based on a database cloud platform of container technology, which can use k8s (kubernetes) or swarm for container management to complete database container deployment and management. Each database instance in the platform is an independent container deployed on the host machine, and a dedicated IP address, a central processing unit (CPU), memory, table space and log space are configured for the container. The table space stores data, and the log space stores database logs. The above-mentioned database refers to all systems that can store data, including relational databases, in-memory databases, non-relational databases, message middleware, etc.
[0070] The target resources of the container may include the CPU, memory, table space, log space, etc. of the container.
[0071] For example, the CPU usage of the container in multiple historical periods can be obtained. The number of historical periods can be determined according to actual conditions. The CPU usage of the container in each historical period can be understood as the ratio of the number of CPUs used in the container in each historical period to the total number of CPUs in the container.
[0072] You can also obtain the usage rate of the container's memory in multiple historical periods. The usage rate of the container's memory in each historical period can be understood as the ratio of the container's memory usage in each historical period to the total amount of the container's memory.
[0073] In this step, after obtaining the usage rate of the target resource of the container in multiple historical periods, the usage rate of the target resource in multiple future periods can be predicted, so that the target resource can be expanded or reduced in time later to improve business efficiency and resource utilization.
[0074] In practical applications, the period may be hours, days, months, etc., and the time interval corresponding to the period may be determined based on actual conditions.
[0075] S102. If the usage rates of consecutive preset cycles in the future are higher than a first threshold, the target resources are expanded; if the usage rates of consecutive preset cycles are lower than a second threshold, the target resources are reduced.
[0076] For example, after predicting the utilization rate of the target resource in multiple future cycles, it is determined whether there are consecutive preset cycles with utilization rate higher than a first threshold. If so, the target resource is expanded; if not, it is determined whether there are consecutive preset cycles with utilization rate lower than a second threshold. When there are consecutive preset cycles with utilization rate lower than the second threshold, the target resource is reduced in capacity; when there are no consecutive preset cycles with utilization rate lower than the second threshold, it is determined that there is no need to expand or reduce the target resource.
[0077] After predicting the usage rate of the target resource in multiple future cycles, it is also possible to determine whether there are consecutive preset cycles with a usage rate lower than a second threshold. If so, the target resource is scaled down. If not, it is determined whether there are consecutive preset cycles with a usage rate higher than the first threshold. If there are consecutive preset cycles with a usage rate higher than the first threshold, the target resource is expanded. If there are no consecutive preset cycles with a usage rate higher than the first threshold, it is determined that there is no need to expand or shrink the target resource.
[0078] The first threshold may be greater than the second threshold, or may be equal to the second threshold. For example, the first threshold is 60% and the second threshold is 30%.
[0079] The continuous preset period can be determined according to actual conditions. For example, when the usage rate of two consecutive periods exceeds a first threshold, the target resource is expanded; when the usage rate of two consecutive periods is lower than a second threshold, the target resource is reduced.
[0080] The database resource scaling method provided in the embodiment of the present application predicts the usage rate of the target resource in the future period, and when the usage rate in the consecutive preset periods exceeds the first threshold, the target resource is expanded, and when the usage rate in the consecutive preset periods is lower than the second threshold, the target resource is reduced, so that the target resource can be expanded or reduced in time, the business efficiency and resource utilization are improved, and the calculation complexity is low and the efficiency is higher.
[0081] Figure 2 A flowchart of a database resource scaling method provided by another embodiment of the present application is shown. Figure 2 As shown, in step S101, obtaining the usage rate of the target resource of the container in multiple historical periods may include the following steps:
[0082] S201. Obtain the target resource currently running in each historical period according to the total usage time of the target resource in each historical period.
[0083] In some embodiments, the target resource is a CPU, and the number of CPUs running in each historical period can be obtained based on the total CPU usage time in each historical period. Specifically, the ratio of the total usage time of the target resource in the historical period to the historical period is calculated to obtain the number of target resources running in the historical period. For example, if the total usage time of the target resource in a historical period is three times that of the historical period, the number of target resources running in the historical period is 3.
[0084] In actual applications, first check the running status of the container. When the running status of the container is normal, perform this step to ensure the accuracy of the calculation results. The running status of the container can be obtained according to the docker command.
[0085] As an implementation method, when the container is running normally, the control group (cgroup) corresponding to the container can be obtained according to the container identification number (ID), and the total usage time of the target resource can be obtained from the control group corresponding to the container. The control group is used to limit, record and isolate the resource usage of the process group, including CPU, memory, disk I / O, etc. The file corresponding to the control group can record the total CPU usage time used by all tasks in the control group and its descendant levels in nanoseconds, so the total CPU usage time can be obtained from the control group corresponding to the container. Correspondingly, the memory usage can also be obtained from the file corresponding to the control group.
[0086] S202: Calculate the usage rate of the target resources in each historical period according to the total amount of the target resources and the target resources being run in each historical period.
[0087] The ratio of the target resource being run in each historical period to the total amount of the target resource may be calculated to obtain the utilization rate of the target resource in each historical period.
[0088] For example, if the target resource is a CPU, the CPU usage rate in each historical period can be calculated based on the number of running CPUs and the total number of CPUs in each historical period.
[0089] In some examples, the total amount of the target resource can be obtained based on the target resource's period and quota, where the period is used to specify how often the container's use of the target resource is reallocated, and the quota refers to the maximum amount of time that can be used to run the container within the target resource's period. The unit of the period and quota is microseconds (μs). For example, the minimum value of the period can be 1000 microseconds, the maximum value can be 1 second (10^6μs), and the default value can be 0.1 second (100000μs). When the quota value is -1, it means no control.
[0090] Specifically, the period and allowed running time of the target resource may be obtained. If the allowed running time of the target resource is greater than zero and the period of the target resource is not equal to zero, the ratio between the allowed running time of the target resource and the period of the target resource is calculated to obtain the total amount of the target resource. If the allowed running time of the target resource is less than or equal to zero, the total amount of the target resource is determined to be the total amount of available target resources (nproc).
[0091] For example, if the target resource is a CPU, the CPU cycle and allowed run time can be obtained. If the CPU allowed run time is greater than zero and the CPU cycle is not equal to zero, the ratio between the CPU allowed run time and the CPU cycle is calculated to obtain the total number of CPUs. If the CPU allowed run time is less than or equal to zero, the total number of CPUs is determined to be the total number of CPUs available in the system.
[0092] In other examples, if the configuration information of the target resource includes the total amount of the target resource, the total amount of the target resource is obtained from the setting information of the target resource, for example, the total number of CPUs is obtained from the CPU configuration information (cpuset), and the total amount of memory is obtained from the memory configuration information.
[0093] In other embodiments, if the target resource is memory, the usage rate of the memory in each historical period may be calculated based on the usage of the memory in each historical period and the total memory of the container.
[0094] As an implementation method, calculating the CPU usage may include:
[0095] a. Check the running status of the container; if the running status of the container is normal, execute step b; if the running status of the container is abnormal, execute step g.
[0096] b. Obtain the identification number of the container; if the identification number of the container is obtained successfully, execute step c; if the identification number of the container fails to be obtained, execute step g.
[0097] c. Obtain the CPU parameters of the container; if the CPU parameters are successfully obtained, execute step d, and if the CPU parameters fail to be obtained, execute step g. The CPU parameters may include the allowed operating time, cycle, and configuration information of the CPU.
[0098] d. Calculate the total number of CPUs; if the total number of CPUs is calculated successfully, execute step e; if the total number of CPUs is calculated unsuccessfully, execute step g.
[0099] e. Get the total CPU usage time; after the total CPU usage time is obtained, execute step f. If the total CPU usage time fails to be obtained, execute step g.
[0100] f. Calculate CPU usage;
[0101] g. Execute at least one of the following methods: report an error, record a log, and exit.
[0102] The log records may include time, operation steps, failure reasons, etc.
[0103] In practical applications, after calculating the usage rate of the target resource in each historical period, the usage rate of the target resource in each historical period can be cached in an array (cached_cpu_usage) for easy access in subsequent use. The data length of the usage rate of the target resource can be determined according to actual conditions.
[0104] The database resource scaling method provided in this embodiment obtains the usage rate of the target resource in each historical period according to the target resource being run in each historical period and the total amount of the target resource, thereby solving the problem of inaccurate resource usage rate acquisition in the prior art.
[0105] Figure 3 A flowchart of a database resource scaling method provided by another embodiment of the present application is shown. Figure 3 As shown, in step S101, predicting the usage rate of the target resource in multiple future collection cycles according to the usage rate of the target resource in multiple historical collection cycles may include the following steps:
[0106] S301. Calculate the average utilization rate of the target resource in multiple historical periods according to the utilization rate of the target resource in multiple historical periods, as the predicted level value of the first period in the future.
[0107] For example, the target resource is CPU. The CPU usage in the first historical cycle is 20%; the CPU usage in the second historical cycle is 24%; the CPU usage in the third historical cycle is 28%; the CPU usage in the fourth historical cycle is 32%; the CPU usage in the fifth historical cycle is 36%; the CPU usage in the sixth historical cycle is 40%; the CPU usage in the seventh historical cycle is 44%; the CPU usage in the eighth historical cycle is 48%; the CPU usage in the ninth historical cycle is 52%; the CPU usage in the tenth historical cycle is 56%; the CPU usage in the eleventh historical cycle is 60%; and the CPU usage in the twelfth historical cycle is 64%.
[0108] Based on the utilization rate of the target resources in the twelve historical cycles, the average utilization rate of the target resources from the first historical cycle to the twelfth historical cycle can be calculated as (20%+24%+28%+32%+36%+40%+44%+48%+52%+56%+60%+64%) / 12=42.
[0109] Then, the average utilization rate of the target resource in multiple historical cycles is used as the predicted level value of the first cycle in the future. It should be noted that a cycle after the last historical cycle in the multiple historical cycles can be the first cycle in the future.
[0110] In some embodiments, it is possible to first determine whether there are enough samples for prediction to improve the accuracy of the prediction. Specifically, it is possible to first check whether the number of usage rates of the target resource in multiple historical periods reaches a preset number. If so, the average usage rate of the target resource in multiple historical periods is calculated based on the usage rate of the target resource in multiple historical periods. The target resource has a usage rate in a historical period, and it is possible to check whether the number of usage rates of the target resource in multiple historical periods reaches a preset number, or it is possible to check whether the number of historical periods reaches a preset number.
[0111] S302. Calculate the predicted level value of the target resource in the future N period based on the smoothing coefficient, the utilization rate of the target resource in the Nth historical period, the predicted level value of the target resource in the future N-1 period, and the predicted trend value of the target resource in the future N-1 period.
[0112] Among them, the first product between the smoothing coefficient and the utilization rate of the target resource in the Nth historical period can be calculated, the first sum between the predicted level value of the target resource in the future N-1th period and the predicted trend value of the target resource in the future N-1th period can be calculated, the second product of the first coefficient and the first sum can be calculated, and the first product and the second product can be added together as the predicted level value of the target resource in the future Nth period. Among them, the sum of the smoothing coefficient and the first coefficient is equal to 1, and N is an integer greater than 1.
[0113] For example, based on the smoothing coefficient α, the utilization rate y2 of the target resource in the second historical period, the predicted level value l1 of the target resource in the first period in the future, and the predicted trend value b1 of the target resource in the first period in the future, the predicted level value l2 of the target resource in the second period in the future is calculated. Specifically, l2=αy2+(1-α)(l1+b1).
[0114] Based on the smoothing coefficient, the utilization rate y3 of the target resource in the third historical period, the predicted level value l2 of the target resource in the second period in the future, and the predicted trend value b2 of the target resource in the second period in the future, the predicted level value l3 of the target resource in the third period in the future is calculated. Specifically, l 3= αy3+(1-α)(l2+b2).
[0115] By analogy, the predicted level values of the target resources in multiple future periods can be calculated.
[0116] S303. Calculate the predicted trend value of the target resource in the future N period based on the trend smoothing coefficient, the predicted level value of the target resource in the future N-1 period, the predicted level value of the target resource in the future N-1 period, and the predicted trend value of the target resource in the future N-1 period.
[0117] Among them, the first difference between the predicted level value of the target resource in the Nth period in the future and the predicted level value of the target resource in the N-1th period in the future can be calculated, and the third product of the first difference and the trend smoothing coefficient can be calculated; the fourth product between the second coefficient and the predicted trend value of the target resource in the N-1th period in the future can be calculated; and the third product and the third product can be added together to obtain the predicted trend value of the target resource in the Nth period in the future. Among them, the sum of the trend smoothing coefficient and the second coefficient is equal to 1.
[0118] For example, based on the trend smoothing coefficient β, the predicted level value l2 of the target resource in the second cycle in the future, the predicted level value l1 of the target resource in the first cycle in the future, and the predicted trend value b1 of the target resource in the first cycle in the future, the predicted trend value b2 of the target resource in the second cycle in the future is calculated. Specifically, b2 = β (l2-l1) + (1-β) b1. Considering that there is not enough data to calculate the initial trend, the predicted trend value of the first cycle in the future can be set to 0, that is, b1 = 0.
[0119] Based on the trend smoothing coefficient β, the predicted level value l3 of the target resource in the third period in the future, the predicted level value l2 of the target resource in the second period in the future, and the predicted trend value b2 of the target resource in the second period in the future, the predicted trend value b3 of the target resource in the third period in the future is calculated. Specifically, b3=β(l3-l2)+(1-β)b2.
[0120] By analogy, the predicted trend value of the target resource in multiple future periods can be calculated.
[0121] Taking the prediction of the next six periods as an example, the target resource is the CPU, the preset smoothing coefficient α=0.5, the trend smoothing coefficient β=0.5, the predicted level value l1=42% for the first future period, the predicted trend value b1=0 for the first future period, and the CPU utilization rate of the second historical period is 24%, the CPU utilization rate of the third historical period is 28%, the CPU utilization rate of the fourth historical period is 32%, the CPU utilization rate of the fifth historical period is 36%, and the CPU utilization rate of the sixth historical period is 40%.
[0122] The predicted CPU level value for the second cycle in the future is:
[0123] l2=αy2+(1-α)(l1+b1)=0.5*24%+(1-0.5)*(42%+0)=33%;
[0124] The predicted trend value of CPU in the second cycle in the future:
[0125] b2=β(l2-l1)+(1-β)b1=0.5*(33%-42%)+(1-0.5)*0=-4.5%.
[0126] The predicted CPU level value for the third cycle in the future is:
[0127] l3=αy3+(1-α)(l2+b2)=0.5*28%+(1-0.5)*(33%-4.5%)=28.25%;
[0128] The predicted trend value of CPU in the next third cycle:
[0129] b3=β(l3-l2)+(1-β)b2=0.5*(28.25%-33%)+(1-0.5)*(-4.5)%=-4.63%.
[0130] The predicted CPU level value for the fourth cycle in the future is:
[0131] l4=αy4+(1-α)(l3+b3)=0.5*32%+(1-0.5)*(28.25%-4.63%)=27.81%;
[0132] The predicted trend value of CPU in the fourth cycle in the future:
[0133] b4=β(l4-l3)+(1-β)b3=0.5*(27.81%-28.25%)+(1-0.5)*(-4.63)%=--2.54%.
[0134] The predicted CPU level value for the fifth cycle in the future is:
[0135] l5=αy5+(1-α)(l4+b4)=0.5*36%+(1-0.5)*(27.81%-2.54%)=30.64%;
[0136] The predicted trend value of CPU in the fifth cycle in the future:
[0137] b5=β(l5-l4)+(1-β)b4=0.5*(30.64%-27.81%)+(1-0.5)*(-2.54)%=0.15%.
[0138] The predicted CPU level value for the sixth cycle in the future is:
[0139] l6=αy6+(1-α)(l5+b5)=0.5*40%+(1-0.5)*(30.64%+0.15%)=35.4%;
[0140] The predicted trend value of CPU in the sixth cycle in the future:
[0141] b6=β(l6-l5)+(1-β) b5=0.5*(35.4%-30.64%)+(1-0.5)*0.15%=2.5%.
[0142] In some embodiments, it is considered that the utilization rate of the target resource may have periodic changes, which is usually caused by the influence of seasons, months, weeks or other time periods. Therefore, seasonal information can also be considered to improve the accuracy of prediction. Specifically, the predicted level value of the target resource in the future Nth cycle can be calculated based on the smoothing coefficient, the utilization rate of the target resource in the Nth historical cycle, the predicted seasonal value of the target resource in the future Nmth cycle, the predicted level value of the target resource in the future N-1th cycle, and the predicted trend value of the target resource in the future Nth cycle, where N is an integer greater than 1, m is an integer greater than 0 and less than N, and m is the length of the seasonal cycle, which can specifically be the data of the previous cycle of the same season. The same season here refers to a repetitive pattern that appears at each fixed time interval in a time series with seasonal characteristics, and each season here can include multiple cycles in this embodiment.
[0143] Specifically, the second difference between the utilization rate of the target resource in the Nth historical period and the predicted seasonal value of the target resource in the Nmth period can be calculated, and the fifth product between the seasonal smoothing coefficient and the second difference can be calculated; the first sum between the predicted level value of the target resource in the future Nth period and the predicted trend value of the target resource in the future N-1th period can be calculated, and the second product of the first coefficient and the first sum can be calculated; then, the fifth product and the second product are added as the predicted level value of the target resource in the future Nth period.
[0144] For example, based on the smoothing coefficient α, the utilization rate y2 of the target resource in the second historical period, the predicted seasonal value s1 of the target resource in the first period in the future, the predicted level value l1 of the target resource in the first period in the future, and the predicted trend value b1 of the target resource in the first period in the future, the predicted level value l2 of the target resource in the second period in the future is calculated. Specifically, l2 = α (y2-s1) + (1-α) (l1 + b1).
[0145] Based on the smoothing coefficient, the utilization rate y3 of the target resource in the third historical period, the predicted seasonal value s2 of the target resource in the second period in the future, the predicted level value l2 of the target resource in the second period in the future, and the predicted trend value b2 of the target resource in the second period in the future, the predicted level value l3 of the target resource in the third period in the future is calculated. Specifically, l3 = α(y3-s3)+(1-α)(l2+b2).
[0146] By analogy, the predicted level values of the target resources in multiple future periods can be calculated.
[0147] In practical applications, there may not be enough data to calculate the initial trend. Therefore, for the initial seasonal component, the average increment of each season can be set to 0, for example, the predicted seasonal value s1 for the first cycle in the future, the predicted seasonal value s2 for the second cycle in the future, and the predicted seasonal value s3 for the third cycle in the future are all 0.
[0148] For other periods with seasonal increments, the utilization rate y of the Nth historical period can be calculated first t and the predicted level value l for the Nth period in the future t , calculate the sixth product between the third difference and the seasonal smoothing coefficient γ; calculate the third coefficient and the predicted seasonal value S of the future Nmth cycle t-m Then, the sixth product and the seventh product are added together to obtain the predicted seasonal value of the target resource in the Nth cycle in the future. t =γ(y t -l t )+(1-γ)s t-m . The sum of the seasonal smoothing coefficient and the third coefficient is equal to 1.
[0149] S304: predict the usage rate of the target resource in the Nth period in the future according to the predicted level value of the target resource in the Nth period in the future and the predicted trend value of the target resource in the Nth period in the future.
[0150] The predicted level value of the target resource in the Nth period in the future and the predicted trend value of the target resource in the Nth period in the future may be added together to obtain the utilization rate of the target resource in the Nth period in the future.
[0151] For example, the predicted level value l1=42% of the CPU in the first cycle in the future and the predicted trend value b1=0 of the CPU in the first cycle in the future are added together, and the predicted CPU usage rate in the first cycle in the future is x1=42%+0=42%;
[0152] Add the predicted level value l2=33% of the CPU in the second cycle in the future and the predicted trend value b2=-4.5% of the CPU in the second cycle in the future, and predict the CPU usage rate x1=33%-4.5%=28.5% in the first cycle in the future;
[0153] The predicted level value l3=28.25% of the CPU in the third cycle in the future and the predicted trend value b3=-4.63% of the CPU in the third cycle in the future are added together, and the predicted CPU usage rate in the third cycle in the future is x4=28.25%-4.63%=23.62%;
[0154] Add the predicted level value l4=27.81% of the CPU in the fourth cycle in the future and the predicted trend value b4=-2.54% of the CPU in the fourth cycle in the future, and predict the CPU usage rate x4=27.81%-2.54%=25.27% in the fourth cycle in the future;
[0155] Add the predicted level value l5=30.64% of the CPU in the fifth cycle in the future and the predicted trend value b5=0.15% of the CPU in the fifth cycle in the future, and predict the CPU usage rate in the fifth cycle in the future x4=30.64%+0.15%=30.79%;
[0156] The predicted level value l6=35.4% of the CPU in the sixth cycle in the future and the predicted trend value b6=2.5% of the CPU in the sixth cycle in the future are added together, and the predicted CPU usage rate in the sixth cycle in the future is x4=35.4%+2.5%=37.9%.
[0157] In some embodiments, if seasonal changes are taken into account, the predicted level value of the target resource in the Nth period in the future, the predicted trend value of the target resource in the Nth period in the future, and the predicted season value of the target resource in the Nth period in the future can be added together to predict the utilization rate of the target resource in the Nth period in the future.
[0158] In practical applications, the utilization rate of target resources in multiple future cycles can be predicted based on the triple exponential smoothing algorithm. Since the usage of a large number of database resources shows obvious trends and / or seasonal patterns, the use of the triple exponential smoothing algorithm can provide relatively accurate short-term predictions, and the prediction results are stable and reliable. In addition, the triple exponential smoothing algorithm can adapt to data changes due to its dynamic coefficients, thus providing more accurate predictions. In addition, the triple exponential smoothing algorithm usually has lower computational complexity, faster calculation speed, lower cost, and is more efficient in practical applications.
[0159] The triple exponential smoothing algorithm can be based on the following recursive relationship:
[0160] l t=α(y t -s t-m )+(1-α)(l t-1 +b t-1 )
[0161] b t =β(l t -l t-1 )+(1-β)b t-1
[0162] s t =γ(y t -l t )+(1-γ)s t-m
[0163] Among them, α is the smoothing parameter, 0<α<1; β is the trend smoothing coefficient, 0<β<1; γ is the seasonal smoothing coefficient, 0<γ<1. t is the smoothed value at time t, y t is the actual value at time t, S t-m is the seasonal value at time tm, l t-1 is the smoothed value at time t-1, b t-1 is the trend value at time t-1, b t-1 is the trend value at time t-1.
[0164] The closer α is to 1, the closer the smoothing value at the current time is to the actual value at the current time, and the less smooth the data is; the closer α is to 0, the closer the smoothing value at the current time is to the smoothing value at the previous time, and the smoother the data is.
[0165] The closer β is to 1, the closer the trend value at time t is to the difference between the smoothed value at time t and the smoothed value at time t-1. The closer β is to 0, the closer the trend value at time t is to the trend value at time t-1.
[0166] The closer γ is to 1, the closer the seasonal value at time t is to the difference between the actual value at time t and the smoothed value at time t. The closer γ is to 0, the closer the seasonal value at time t is to the seasonal value at time tm.
[0167] According to the recursive relationship of exponential smoothing, in the exponential smoothing algorithm, the previous actual values have an impact on the current smoothed value, but the role of the previous actual values will gradually decrease as the power of the parameter increases, and the role of the relatively earlier actual values is relatively small. Therefore, as long as a few time steps have passed, the impact of the initial value is very small, so the initial value can be set to 0, for example, the predicted trend value of the first cycle in the future is 0, and the predicted seasonal value of the first cycle in the future is 0.
[0168] Among them, the recursive relationship of exponential smoothing is as follows:
[0169]
[0170] As an implementation method, predicting the usage rate of the target resource in multiple future cycles based on the usage rate of the target resource in multiple historical cycles may include:
[0171] a. Check whether the number of usage rates of the target resource in multiple historical periods reaches a preset number; if so, execute step b; if not, end directly.
[0172] b. Set the smoothing coefficient; if the smoothing coefficient is set successfully, execute step c; if the smoothing coefficient fails to be set, execute step g.
[0173] c. Set the trend smoothing coefficient; if the trend smoothing coefficient is set successfully, execute step d; if the trend smoothing coefficient fails to be set, execute step g.
[0174] d. Set the seasonal smoothing coefficient; if the seasonal smoothing coefficient is set successfully, execute step e; if the seasonal smoothing coefficient is set unsuccessfully, execute step g.
[0175] e. Calculate the average utilization rate of the target resource in multiple historical cycles as the predicted level value for the first cycle in the future; after the calculation is completed, execute step f; if the calculation fails, execute step g.
[0176] f. Cyclic prediction of the utilization rate of target resources in multiple future cycles;
[0177] g. Execute at least one of the following methods: report an error, record a log, and exit.
[0178] In practical applications, after predicting the usage of the target resource in multiple cycles in the future, the usage of the target resource in multiple cycles in the future can be cached in an array (pred_cpu_usage) for easy access in subsequent use. The data length of the usage of the target resource can be determined according to actual conditions.
[0179] The database resource scaling method provided in this embodiment predicts the usage rate of the target resource in multiple future cycles, has high prediction accuracy and low computational complexity.
[0180] Figure 4 A flowchart of a database resource scaling method provided by another embodiment of the present application is shown. Figure 4 As shown, in step S102, if the usage rate of the consecutive preset periods is higher than the first threshold, the target resource is expanded, which may include the following steps:
[0181] S401 , determining whether there are consecutive preset periods of usage rates higher than a first threshold among the usage rates of multiple future periods.
[0182] In some embodiments, it may be determined first whether there is sufficient forecast data to more accurately determine whether capacity expansion is required. Specifically, it may be determined first whether the number of utilization rates in multiple future cycles reaches a preset threshold, and if so, it may be determined whether there are continuous preset cycles of utilization rates higher than a first threshold among the utilization rates in multiple future cycles. Wherein, the target resource has a utilization rate in a future cycle, and it may be determined whether the number of utilization rates in multiple future cycles reaches a preset threshold, or it may be determined whether the number of multiple future cycles reaches a preset threshold.
[0183] In practical applications, the following formula can be used to check whether there is a usage rate higher than the first threshold for consecutive preset periods:
[0184]
[0185] Among them, T expansion is the first threshold, k is the last index of the current window, and n is the number of consecutive preset cycles.
[0186] If so, execute step S402, if not, determine that capacity expansion is not required.
[0187] S402: Calculate a first percentage of the portion of the usage rate of consecutive preset periods that exceeds a first threshold.
[0188] Specifically, the portion of the usage rate of each period in the consecutive preset periods that exceeds the first threshold can be added to obtain a second sum, and the eighth product of the number of consecutive preset periods and the first threshold can be calculated, and then the first percentage of the second sum and the eighth product can be calculated.
[0189] In practical applications, the first percentage of the portion where the usage rate of consecutive preset periods exceeds the first threshold can be calculated by the following formula:
[0190]
[0191] For example, the multiple future cycles include five cycles, and the predicted CPU usages of the five cycles are 52%, 56%, 61%, 66%, and 22.75%, respectively, and the first threshold is 60%. The usages of the third and fourth cycles exceed the first threshold, and the first percentage of the portion of the usages of the third and fourth cycles exceeding the first threshold is calculated:
[0192] [(61%+66%)-2*60%] / 2*60%=5.8%.
[0193] S403: Calculate the expansion capacity of the target resources according to the first percentage and the total amount of the target resources.
[0194] Specifically, the first percentage is multiplied by the total amount of the target resource to obtain the expansion capacity of the target resource.
[0195] For example, the configuration information of the current database can be obtained, and the total amount of the target resource can be obtained from the configuration information. For example, if the target resource is a CPU and the package of the current database is 16C 64G, it can be known that the total amount of CPUs is 16. Then, according to the first percentage of 5.8% and the total amount of CPUs being 16, the CPU expansion capacity can be calculated to be 0.928, and after rounding up, it is determined that the CPU expansion capacity is 1.
[0196] As an implementation method, determining the expansion capacity of the target resource may include:
[0197] a. Check whether the number of usage rates in the future multiple cycles reaches the preset threshold; if so, execute step b; if not, execute step f.
[0198] b. Check whether there is a usage rate higher than the first threshold for consecutive preset periods; if so, execute step c; if not, execute step f.
[0199] c. Calculate the first percentage of the usage rate of consecutive preset periods exceeding the first threshold; if the calculation is successful, execute step d; if the calculation fails, execute step f.
[0200] d. Obtain the configuration information of the current database, and obtain the total amount of the target resource from the configuration information; if the acquisition is successful, execute step e; if the acquisition fails, execute step f.
[0201] e. Calculate the expansion capacity of the target resource based on the first percentage and the total amount of the target resource; after the calculation is completed, end; if the calculation fails, execute step f.
[0202] f. Execute at least one of the following methods: report an error, record a log, and exit.
[0203] S404: Expand the target resource according to the expansion capacity of the target resource.
[0204] In some embodiments, it is determined whether the number of expansions within a preset time has reached the preset number of expansions. If not, it means that the number of expansions within the preset time has not reached the configured upper limit, and expansion can continue. Therefore, when the number of expansions within the preset time has not reached the preset number of expansions, the target resource can be expanded according to the expansion capacity and the cycle of the target resource, or the number of cores matching the expansion capacity can be increased for the target resource.
[0205] In actual applications, you can query the expansion work order records to obtain the number of expansions of the target resource of the database service within the preset time, and compare the query results with the configured preset expansion times. If the number of expansions of the target resource within the preset time is less than the preset expansion times, it means that the number of automatic expansions within the preset time has not reached the configured upper limit, and expansion can continue; if the number of expansions of the target resource within the preset time is greater than or equal to the preset expansion times, it means that the number of automatic expansions within the preset time has reached the configured upper limit, and the exception processing is performed and ended.
[0206] In some examples, when the number of expansions within a preset time does not reach a preset number of expansions, the running status of the container and the database status can be checked. If the running status of the container and the database status are normal, then the target resource can be expanded.
[0207] For example, you can use the docker command to get the running status of the container. If the running status of the container is abnormal and cannot be expanded, you can handle the exception and end it.
[0208] Whether the database is normal can be determined by whether the database is connected or whether the process exists. For example, the client can be used to connect to the redis service (server) and execute to view information (info). If the execution is successful, redis is normal, otherwise it is abnormal. The redis service can be connected and the information can be viewed by the redis IP address, port and password. If the database is abnormal, expansion cannot be performed, and exception handling is required and the process ends.
[0209] In some embodiments, the target resource is a CPU, and the container CPU may be expanded in different ways depending on the binding method. The specific method of expanding the capacity may include a binding cycle limit and a binding core number limit. Expanding the CPU by binding the cycle limit may specifically be to calculate the product between the CPU expansion capacity (the number of expansions) and the CPU cycle, and obtain the CPU expansion time to increase the CPU running time. Expanding the CPU by binding the core number limit may be to allocate a number of cores to the container that matches the CPU expansion capacity.
[0210] In actual applications, the data and log space used by the container does not need to be expanded through the container, but the directory expansion needs to be performed on the server where the container is located.
[0211] In some examples, you can also expand the memory of the database. For example, to expand the memory of the MySQL database, you need to modify the size of the database buffer pool after expanding the container memory to increase the amount of data cache and improve database performance. To expand the memory of the Redis database, you need to modify the maximum memory of the database after expanding the container memory to increase the upper limit of the cache amount.
[0212] As an implementation method, expanding the capacity of target resources may include:
[0213] a. Determine whether the number of expansions within a preset time has reached the preset number of expansions; if not, execute step b; if so, execute step f.
[0214] b. Check the running status of the container; if the running status of the container is normal, execute step c; if the running status of the container is abnormal, execute step f.
[0215] c. Check the status of the database; if the status of the database is normal, execute step d; if the status of the database is abnormal, execute step f.
[0216] d. Expand the target resource; if the expansion is successful, execute step e; if the expansion fails, execute step f.
[0217] e. Expand the database. If the expansion is successful, the process ends. If the expansion fails, execute step f.
[0218] f. Execute at least one of the following methods: report an error, record a log, and exit.
[0219] This embodiment provides a database scaling method, which timely expands the capacity of target resources when the usage rate of consecutive preset periods exceeds a first threshold, thereby improving business efficiency and resource utilization, and has low computational complexity.
[0220] Figure 5 A flowchart of a database resource scaling method provided by another embodiment of the present application is shown. Figure 5 As shown, in step S102, if the usage rate of the consecutive preset periods is lower than the second threshold, the target resource is scaled down, which may include the following steps:
[0221] S501. Determine whether there are consecutive preset periods of usage rates lower than a second threshold value among the usage rates of multiple future periods.
[0222] In some embodiments, it may be determined whether there is sufficient prediction data to more accurately determine whether capacity reduction is required. Specifically, it may be determined whether the number of utilization rates in multiple future cycles reaches a preset threshold, and if so, it may be determined whether there are continuous preset cycles of utilization rates lower than a second threshold among the utilization rates in multiple future cycles.
[0223] If so, execute step S502, if not, determine that scaling down is not necessary.
[0224] S502: Calculate a second percentage of the portion of consecutive preset periods in which the usage rate is lower than a second threshold.
[0225] Specifically, the third sum can be obtained by adding the portion of the usage rate of each cycle in the consecutive preset cycles that is lower than the second threshold, and the ninth product of the number of consecutive preset cycles and the second threshold is calculated, and then the second percentage of the third sum and the ninth product is calculated. It should be noted that the second percentage is a negative number.
[0226] S503: Calculate the reduced capacity of the target resource according to the second percentage and the total amount of the target resource.
[0227] Specifically, the absolute value of the second percentage is multiplied by the total amount of the target resource to obtain the reduced capacity of the target resource.
[0228] As an implementation method, determining the expansion capacity of the target resource may include:
[0229] a. Check whether the number of usage rates in the future multiple cycles reaches the preset threshold; if so, execute step b; if not, execute step f.
[0230] b. Check whether there is a continuous preset period of usage rate lower than the second threshold; if so, execute step c, if not, execute step f.
[0231] c. Calculate the second percentage of the portion of the usage rate of the consecutive preset cycles that is lower than the second threshold; if the calculation is successful, execute step d; if the calculation fails, execute step f.
[0232] d. Obtain the configuration information of the current database, and obtain the total amount of the target resource from the configuration information; if the acquisition is successful, execute step e; if the acquisition fails, execute step f.
[0233] e. Calculate the reduced capacity of the target resource based on the second percentage and the total amount of the target resource; after the calculation is completed, end; if the calculation fails, execute step f.
[0234] f. Execute at least one of the following methods: report an error, record a log, and exit.
[0235] S504: Reduce the capacity of the target resource according to the reduced capacity of the target resource.
[0236] In some embodiments, it is determined whether the number of shrinking times within a preset time has reached the preset shrinking number. If not, it means that the number of shrinking times within the preset time has not reached the configured upper limit, and the shrinking can continue. Therefore, when the number of shrinking times within the preset time has not reached the preset shrinking number, the target resource can be shrunk according to the shrinking capacity and the cycle of the target resource, or the number of cores matching the shrinking capacity can be increased for the target resource.
[0237] In actual applications, you can query the shrinking work order records to obtain the number of shrinking times of the target resource of the database service within the preset time, and compare the query results with the configured preset shrinking times. If the number of shrinking times of the target resource within the preset time is less than the preset shrinking times, it means that the number of automatic shrinking times within the preset time has not reached the configured upper limit, and the shrinking can continue; if the number of shrinking times of the target resource within the preset time is greater than or equal to the preset shrinking times, it means that the number of automatic shrinking times within the preset time has reached the configured upper limit, and the exception processing is performed and ended.
[0238] In some examples, when the number of shrinking times within a preset time does not reach a preset number of shrinking times, the running status of the container and the database status can be checked. If the running status of the container and the database status are normal, then the target resource can be shrunk.
[0239] For example, you can use the docker command to get the running status of the container. If the running status of the container is abnormal and cannot be reduced, you can handle the exception and end it.
[0240] Whether the database status is normal can be determined by whether the database is connected or whether the process exists. For example, the client can be used to connect to the redis service (server) and execute to view information (info). If the execution is successful, redis is normal, otherwise it is abnormal. The redis service can be connected and the information can be viewed by the redis IP address, port and password. If the database status is abnormal, it cannot be scaled down, and the exception needs to be handled and ended.
[0241] In some embodiments, the target resource is a CPU, and the specific method of shrinking the container CPU may include a binding cycle limit and a binding core number limit depending on the binding method. Scaling the CPU by binding cycle limit may specifically be calculating the product between the CPU's shrinking capacity (the number of shrinkages) and the CPU's cycle to obtain the CPU's shrinking time to increase the CPU's running time. Scaling the CPU by binding core number limit may be allocating a number of cores to the container that matches the shrinking capacity of the CPU.
[0242] In actual applications, the data and log space used by the container does not need to be reduced through the container. Instead, the directory needs to be reduced on the server where the container is located.
[0243] In some examples, you can also reduce the memory of the database. For example, to reduce the memory of the MySQL database, you need to modify the size of the database buffer pool after reducing the memory of the container to increase the amount of data cache and improve the performance of the database. To reduce the memory of the Redis database, you need to modify the maximum memory of the database after reducing the memory of the container to increase the upper limit of the amount of cache.
[0244] As an implementation method, scaling down the target resource may include:
[0245] a. Determine whether the number of shrinkages within a preset time has reached a preset number of shrinkages; if not, execute step b; if so, execute step f.
[0246] b. Check the running status of the container; if the running status of the container is normal, execute step c; if the running status of the container is abnormal, execute step f.
[0247] c. Check the status of the database; if the status of the database is normal, execute step d; if the status of the database is abnormal, execute step f.
[0248] d. Scale down the target resource; if the scale-down is successful, execute step e; if the scale-down fails, execute step f.
[0249] e. Reduce the size of the database. If the reduction is successful, the process ends. If the reduction fails, execute step f.
[0250] f. Execute at least one of the following methods: report an error, record a log, and exit.
[0251] This embodiment provides a database scaling method, which timely scales down target resources when the usage rate for consecutive preset periods is lower than a human threshold, reduces manual participation, improves business efficiency and resource utilization, and has low computational complexity.
[0252] Figure 6 A schematic diagram of the structure of a database resource expansion device provided by an embodiment of the present application is shown. Figure 6 As shown, the database resource expansion and contraction device 10 of this embodiment is used to implement the operation corresponding to the electronic device in any of the above method embodiments. The database resource expansion and contraction device 10 of this embodiment includes:
[0253] The first processing module 11 is used to obtain the usage rate of the target resource of the container in multiple historical periods, and predict the usage rate of the target resource in multiple future periods according to the usage rate of the target resource in multiple historical periods;
[0254] The second processing module 12 is used to expand the target resource if the usage rate of the consecutive preset cycles is higher than the first threshold value among the usage rates of the multiple future cycles, and to reduce the target resource if the usage rate of the consecutive preset cycles is lower than the second threshold value.
[0255] In some embodiments, the first processing module 11 includes a collection module 111 and a prediction module 112. The collection module 111 can obtain the target resources running in each historical period based on the total usage time of the target resources in each historical period; and calculate the usage rate of the target resources in each historical period based on the total amount of the target resources and the target resources running in each historical period.
[0256] In practical applications, the acquisition module 111 may receive a periodic execution instruction from a computer system (agent), which carries the acquisition cycle, the cache quantity, and the monitoring object, such as the CPU, memory, directory space, etc. After receiving the reverse periodic execution instruction, the acquisition module 111 may obtain the target resource running in each historical period according to the total usage time of the target resource in each historical period; and calculate the usage rate of the target resource in each historical period according to the total amount of the target resource and the target resource running in each historical period.
[0257] The prediction module 112 can calculate the average utilization rate of the target resource in multiple historical cycles according to the utilization rate of the target resource in multiple historical cycles as the predicted level value of the first future cycle; calculate the predicted level value of the target resource in the future N cycle based on the smoothing coefficient, the utilization rate of the target resource in the Nth historical cycle, the predicted level value of the target resource in the future N-1th cycle, and the predicted trend value of the target resource in the future N-1th cycle; N is an integer greater than 1; calculate the predicted trend value of the target resource in the future N cycle based on the trend smoothing coefficient, the predicted level value of the target resource in the future N-1th cycle, the predicted level value of the target resource in the future N-1th cycle, and the predicted trend value of the target resource in the future N-1th cycle; calculate the utilization rate of the target resource in the future N cycles according to the predicted level value of the target resource in the future N cycle and the predicted trend value of the target resource in the future N cycle.
[0258] In practical applications, the prediction module 112 may also receive periodic execution instructions from a computer system (agent), and the periodic execution instructions may include the cache quantity, the predicted usage rate in the next P cycles, the smoothing coefficient, the trend smoothing coefficient, the seasonal smoothing coefficient, and the seasonal cycle length. After receiving the periodic execution instructions, the prediction module 112 may predict the usage rate of the target resource in the next N cycles.
[0259] In some embodiments, the second processing module 12 may include a decision module 121, an expansion module 122, and a reduction module 123. The decision module 121 is used to determine whether, among the utilization rates of the multiple future cycles, there are continuous preset cycles with utilization rates higher than a first threshold value; if so, calculate the first percentage of the portion of the utilization rates of the continuous preset cycles that exceeds the first threshold value; and calculate the expansion capacity of the target resource based on the first percentage and the total amount of the target resource. The decision module 122 is also used to determine whether, among the utilization rates of the multiple future cycles, there are continuous preset cycles with utilization rates lower than a second threshold value; if so, calculate the second percentage of the portion of the utilization rates of the continuous preset cycles that are lower than the second threshold value; and calculate the reduction capacity of the target resource based on the second percentage and the total amount of the target resource.
[0260] Correspondingly, the capacity expansion module 122 is used to expand the target resource according to the expansion capacity of the target resource, and the capacity reduction module 123 is used to reduce the capacity of the target resource according to the reduction capacity of the target resource.
[0261] The database resource expansion device 10 provided in the embodiment of the present application can execute the above method embodiment. Its specific implementation principle and technical effects can be found in the above method embodiment, which will not be described in detail in this embodiment.
[0262] Figure 7 FIG. 1 shows a hardware structure diagram of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 20 is used to implement the operations corresponding to the electronic device in any of the above method embodiments. The electronic device 20 of this embodiment may include: a memory 21, a processor 22 and a communication interface 23.
[0263] The memory 21 is used to store computer instructions. The memory 21 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk.
[0264] The processor 22 is used to execute the computer instructions stored in the memory to implement the method in the above embodiment. For details, please refer to the relevant description in the above method embodiment. The processor 22 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.
[0265] Optionally, the memory 21 may be independent or integrated with the processor 22 .
[0266] The communication interface 23 may be connected to the processor 22. The processor 22 may control the communication interface 23 to implement the functions of receiving and sending signals.
[0267] The electronic device provided in this embodiment can be used to execute the above-mentioned database resource scaling method, and its implementation method and technical effect are similar, which will not be described in detail in this embodiment.
[0268] The present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the methods provided in the various embodiments described above.
[0269] The present application also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. At least one processor of a device can read the computer instructions from the computer-readable storage medium, and at least one processor executes the computer instructions so that the device implements the methods provided in the various embodiments described above.
[0270] An embodiment of the present application also provides a chip, which includes a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call and execute the computer instructions from the memory, so that a device equipped with the chip executes the methods described in the various possible implementation modes above.
[0271] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for scaling database resources, characterized in that: The method comprises: Obtaining usage rates of target resources of the container in multiple historical periods, and predicting usage rates of the target resources in multiple future periods based on the usage rates of the target resources in multiple historical periods; Among the utilization rates of the future multiple cycles, if the utilization rates of consecutive preset cycles are higher than a first threshold, the target resource is expanded; if the utilization rates of consecutive preset cycles are lower than a second threshold, the target resource is reduced.
2. The method according to claim 1, characterized in that The obtaining of the usage rate of the target resource of the container in multiple historical periods specifically includes: According to the total usage time of the target resource in each of the historical periods, obtaining the target resource being run in each of the historical periods; The utilization rate of the target resource in each historical period is calculated according to the total amount of the target resource and the target resource being operated in each historical period.
3. The method according to claim 2, characterized in that The method for obtaining the total amount of the target resource includes: Obtaining the cycle and allowed running time of the target resource; If the allowed running time of the target resource is greater than zero and the period of the target resource is not equal to zero, calculating the ratio between the allowed running time of the target resource and the period of the target resource to obtain the total amount of the target resource; If the allowed running time of the target resource is less than or equal to zero, determining the total amount of the target resource as the total amount of available target resources; Alternatively, when the setting information of the target resource includes the total amount of the target resource, the total amount of the target resource is acquired from the setting information of the target resource.
4. The method according to claim 1, characterized in that: The predicting the usage rate of the target resource in multiple future cycles according to the usage rate of the target resource in multiple historical cycles specifically includes: Calculating the average utilization rate of the target resource in multiple historical periods according to the utilization rate of the target resource in multiple historical periods as the predicted level value of the first future period; Based on the smoothing coefficient, the utilization rate of the target resource in the Nth historical period, the predicted level value of the target resource in the N-1th period in the future, and the predicted trend value of the target resource in the N-1th period in the future, the predicted level value of the target resource in the N-1th period in the future is calculated; N is an integer greater than 1; Calculate the predicted trend value of the target resource in the Nth period in the future based on the trend smoothing coefficient, the predicted level value of the target resource in the Nth period in the future, the predicted level value of the target resource in the N-1th period in the future, and the predicted trend value of the target resource in the N-1th period in the future; The utilization rate of the target resource in the next N cycles is calculated according to the predicted level value of the target resource in the next N cycles and the predicted trend value of the target resource in the next N cycles.
5. The method according to claim 4, characterized in that The plurality of historical cycles include cross-seasonal cycles; The calculating of the predicted level value of the target resource in the future N-th period based on the smoothing coefficient, the utilization rate of the target resource in the N-th historical period, the predicted level value of the target resource in the future N-1-th period, and the predicted trend value of the target resource in the future N-1-th period specifically includes: Based on the smoothing coefficient, the utilization rate of the target resource in the Nth historical period, the predicted seasonal value of the target resource in the future Nmth period, the predicted level value of the target resource in the future N-1th period, and the predicted trend value of the target resource in the future N-1th period, the predicted level value of the target resource in the future N-1th period is calculated.
6. The method according to claim 4 or 5, characterized in that: The calculating the average usage rate of the target resource in multiple historical periods according to the usage rate of the target resource in multiple historical periods specifically includes: Checking whether the number of utilization rates of the target resource in multiple historical periods reaches a preset number; If so, the average utilization rate of the target resource in the multiple historical periods is calculated according to the utilization rate of the target resource in the multiple historical periods.
7. The method according to any one of claims 1 to 5, characterized in that: If the usage rate of the consecutive preset periods is higher than the first threshold, the target resource is expanded, specifically including: Determine whether among the usage rates of the future multiple periods, there are consecutive preset periods with usage rates higher than a first threshold; If so, calculating a first percentage of the usage rate of the consecutive preset periods exceeding the first threshold; Calculating an expansion capacity of the target resource according to the first percentage and the total amount of the target resource; The target resource is expanded according to the expansion capacity of the target resource.
8. The method according to claim 7, characterized in that The expanding the capacity of the target resource according to the expanded capacity of the target resource specifically includes: Determine whether the number of expansion times within a preset time has reached a preset number of expansion times; If not, the target resource is expanded according to the expansion capacity and the cycle of the target resource, or the number of cores matching the expansion capacity is increased for the target resource.
9. The method according to any one of claims 1 to 5, characterized in that: If the usage rate of the consecutive preset periods is lower than the second threshold, scaling down the target resource specifically includes: Determine whether there are consecutive preset periods of usage rates lower than a second threshold value among the usage rates of the future multiple periods; If so, calculating a second percentage of the portion of the continuous preset period whose usage rate is lower than the second threshold; Calculating a reduced capacity of the target resource according to the second percentage and the total amount of the target resource; The target resource is reduced in capacity according to the reduced capacity of the target resource.
10. The method according to claim 9, characterized in that The reducing the capacity of the target resource according to the reduced capacity of the target resource specifically includes: Determine whether the number of shrinkages within a preset time has reached a preset number of shrinkages; If not, the target resource is scaled down according to the scaled down capacity and the period of the target resource, or the number of cores for the target resource is reduced to match the scaled down capacity.
11. A database resource expansion device, characterized in that: The device comprises: A first processing module is used to obtain the usage rate of the target resource of the container in multiple historical periods, and predict the usage rate of the target resource in multiple future periods according to the usage rate of the target resource in multiple historical periods; The second processing module is used to expand the target resource if the usage rate of the consecutive preset cycles is higher than the first threshold value among the usage rates of the multiple future cycles, and to reduce the target resource if the usage rate of the consecutive preset cycles is lower than the second threshold value.
12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.
14. A computer program product, characterized in that The computer program product comprises a computer program, which implements the method according to any one of claims 1 to 10 when executed by a processor.
Citation Information
Patent Citations
Application container adjusting method, adjusting system, computer readable medium and terminal equipment
CN111522636A
Resource management method and device, equipment and storage medium
CN117544635A
Cluster dynamic scaling method and system based on periodic data prediction
CN118467098A
