Capacity management method and related device

Abstract

The present disclosure provides a capacity management method and related equipment, and relates to the technical field of big data. The method comprises the following steps: acquiring a used resource amount of a first cluster, a maximum resource amount of the first cluster, and historical information corresponding to a second cluster, the first cluster and the second cluster being different server clusters corresponding to the same user; predicting a resource amount growth rate of the first cluster according to multiple resource amount data; determining a remaining available time of the first cluster according to the used resource amount, the maximum resource amount and the resource amount growth rate, the remaining available time being a ratio of a resource amount difference and the resource amount growth rate, the resource amount difference being a difference between the maximum resource amount and the used resource amount; and outputting an alarm information in the case that the remaining available time is less than an alarm time. The present disclosure can avoid the business interference problem caused by limiting the write operation of the user, and the robustness of the service running in the current server cluster is improved.

Description

Technical Field

[0001] This disclosure relates to the technical field of big data, specifically to a capacity management method and related equipment. Background Technology

[0002] When managing HBase cluster capacity using related technologies, a capacity threshold is set for the cluster when creating it for a user. When the used capacity of the cluster exceeds this threshold, the server will issue a prompt message to remind the user to expand the current cluster. During the cluster expansion process, the cluster's external services will be suspended, which will cause the user's business running within the cluster to be hindered. In other words, when managing HBase clusters using related technologies, the robustness of the services running on the cluster is relatively low. Summary of the Invention

[0003] The purpose of this disclosure is to provide a capacity management method and related equipment to solve the technical problem of low robustness of services running on the HBase cluster when managing HBase clusters using related technologies.

[0004] In a first aspect, this disclosure provides a capacity management method, including:

[0005] The system obtains the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster. The first cluster and the second cluster are different server clusters corresponding to the same user. The activation time of the first cluster is later than that of the second cluster. The historical information includes multiple resource data, which are used to indicate the amount of resources used by the second cluster in the corresponding historical period.

[0006] Based on the multiple resource quantity data, predict the resource quantity growth rate of the first cluster;

[0007] The remaining available time of the first cluster is determined based on the amount of resources used, the maximum amount of resources, and the resource growth rate. The remaining available time is the ratio of the resource difference to the resource growth rate. The resource difference is the difference between the maximum amount of resources and the amount of resources used.

[0008] If the remaining available time is less than the warning time, an alarm message is output to remind the user to expand the capacity of the first cluster.

[0009] In one embodiment, predicting the resource growth rate of the first cluster based on the plurality of resource quantity data includes:

[0010] The resource quantity data are analyzed based on a pre-acquired target regression model to obtain the resource quantity growth rate, wherein the target regression model is a neural network model.

[0011] In one embodiment, before predicting the resource growth rate of the first cluster based on the plurality of resource quantity data, the method further includes:

[0012] Multiple sets of training data are acquired, wherein each set of training data includes a first parameter and multiple second parameters. The first parameter indicates the growth rate of the amount of resources used by the corresponding cluster, and the second parameter indicates the amount of resources used by the corresponding cluster in a historical period. Different second parameters in each set of training data correspond to different historical periods.

[0013] The initial regression model is trained based on the multiple training data to obtain the target regression model.

[0014] In one embodiment, before obtaining the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster, the method further includes:

[0015] Deploy the first sub-cluster for storing data, and the second sub-cluster for containerizing cluster components;

[0016] Build multiple component containers in the second sub-cluster;

[0017] The first cluster is generated based on the first sub-cluster and the multiple component containers.

[0018] In one embodiment, the historical information further includes multiple expansion waiting times, wherein the expansion waiting time is the difference between a first time and a second time, the second time is the generation time of the capacity warning information of the second cluster, and the first time is the generation time of the expansion information in response to the capacity warning information;

[0019] After obtaining the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster, the method further includes:

[0020] The average of the multiple expansion waiting times is determined as the target waiting time;

[0021] The alert time is determined based on the target waiting time and the preset expansion time, wherein the expansion time is the time consumed in expanding the first cluster.

[0022] In one embodiment, after outputting the alarm information, the method further includes:

[0023] Obtain the expansion command corresponding to the alarm information;

[0024] According to the expansion instruction, the first cluster is expanded to increase the maximum resource volume of the first cluster.

[0025] In one embodiment, the scaling up process for the first cluster according to the scaling up instruction includes:

[0026] According to the expansion instruction, the maximum computing resources of the first cluster are increased, and / or the maximum storage resources of the first cluster are increased.

[0027] Secondly, this disclosure also provides a capacity management device, comprising:

[0028] The acquisition module is used to acquire the amount of resources used by the first cluster, the maximum amount of resources of the first cluster, and the historical information of the corresponding second cluster. The first cluster and the second cluster are different server clusters corresponding to the same user. The activation time of the first cluster is later than the activation time of the second cluster. The historical information includes multiple resource data, which are used to indicate the amount of resources used by the second cluster in the corresponding historical period.

[0029] The prediction module is used to predict the resource growth rate of the first cluster based on the multiple resource quantity data.

[0030] The determination module is used to determine the remaining available time of the first cluster based on the amount of resources used, the maximum amount of resources, and the resource growth rate. The remaining available time is the ratio of the resource difference to the resource growth rate, and the resource difference is the difference between the maximum amount of resources and the amount of resources used.

[0031] The output module is used to output alarm information when the remaining available time is less than the warning time, and the alarm information is used to remind the user to expand the capacity of the first cluster.

[0032] Thirdly, this disclosure also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the above-described capacity management method.

[0033] Fourthly, this disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described capacity management method.

[0034] In this disclosure, based on the resource usage data of the user's server cluster in historical time periods, the resource growth rate of the user's current server cluster is predicted. Combined with the current usage and maximum resource capacity of the user's current server cluster, the remaining available time of the user's current server cluster is calculated. When the remaining available time is insufficient, an alarm message is output to remind the user to expand the currently used server cluster in a timely manner. Since the remaining available resources of the user's current server cluster are relatively sufficient when the alarm message is output, the expansion operation of the server cluster can be supported without restricting the user's write operations. This can avoid the business interference problem caused by restricting the user's write operations and improve the robustness of the services running in the current server cluster. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of a capacity management method provided in an embodiment of this disclosure;

[0036] Figure 2 This is a schematic diagram of a regression model provided in an embodiment of this disclosure;

[0037] Figure 3 This is a schematic diagram of a cluster deployment process provided in an embodiment of this disclosure;

[0038] Figure 4 This is a schematic diagram of a capacity management device provided in an embodiment of this disclosure;

[0039] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0040] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0041] This disclosure provides a capacity management method, such as... Figure 1 As shown, the capacity management method includes:

[0042] Step 101: Obtain the amount of resources used in the first cluster, the maximum amount of resources in the first cluster, and the historical information of the corresponding second cluster.

[0043] The first cluster and the second cluster are different server clusters corresponding to the same user. The first cluster was started later than the second cluster. The historical information includes multiple resource quantity data, which are used to indicate the amount of resources used by the second cluster in the corresponding historical period.

[0044] The method described in this disclosure is applied to a cloud service platform. The cloud service platform allocates a namespace to each user (also known as a tenant) who accesses the platform, and then manages the capacity of the namespace (such as computing resources, storage resources, etc.). By deploying a server cluster within the namespace, the corresponding user can use it. That is, the user can run the required business services in the server cluster within the namespace.

[0045] For multiple users on the platform, each user has a different namespace. However, for the same user, the namespace can be the same or different each time they access the platform and rent cloud service resources. That is, the platform will allocate a namespace to the user during each rental. When the user cancels the rental of cloud server resources, the platform will reclaim the previously allocated namespace. When the user accesses the platform again and rents cloud service resources, the platform will reassign a new namespace to the user. The two allocated namespaces may be the same or different.

[0046] The first cluster can be understood as the server cluster rented by the target user when accessing the platform, while the second cluster is the server cluster rented by the target user when accessing the platform previously. The target user is one of multiple users who have accessed the platform and rented server clusters.

[0047] It should be noted that, in this disclosure, the second cluster corresponding to the historical information can be one, two or more, and the more second clusters corresponding to the historical information, the more accurate the predicted resource growth rate of the first cluster will be.

[0048] The historical information includes multiple resource data points that correspond to multiple consecutive historical time periods. For example, the multiple resource data points included in the historical information may be: the resource amount used by the target user when renting the second cluster on Monday of the previous week, the resource amount used by the target user when renting the second cluster on Tuesday of the previous week, the resource amount used by the target user when renting the second cluster on Wednesday of the previous week, the resource amount used by the target user when renting the second cluster on Thursday of the previous week, the resource amount used by the target user when renting the second cluster on Friday of the previous week, the resource amount used by the target user when renting the second cluster on Saturday of the previous week, and the resource amount used by the target user when renting the second cluster on Sunday of the previous week.

[0049] Step 102: Based on the multiple resource quantity data, predict the resource quantity growth rate of the first cluster.

[0050] By analyzing the resource usage of the target user's server cluster over multiple consecutive historical periods, the growth rate of the target user's resource usage when renting a server cluster can be predicted.

[0051] For example, curve fitting can be applied to process the multiple resource quantity data to predict the resource quantity growth rate of the first cluster; or model learning can be applied to process the multiple resource quantity data to predict the resource quantity growth rate of the first cluster.

[0052] Step 103: Determine the remaining available time of the first cluster based on the amount of resources used, the maximum amount of resources, and the resource growth rate.

[0053] Wherein, the remaining available time is the ratio of the resource quantity difference to the resource quantity growth rate, and the resource quantity difference is the difference between the maximum resource quantity and the used resource quantity.

[0054] Step 104: If the remaining available time is less than the warning time, output an alarm message.

[0055] The alarm information is used to remind the user to expand the capacity of the first cluster.

[0056] In related technologies, after allocating a namespace to the target user and deploying a server cluster within that namespace, a capacity threshold is set accordingly. When the amount of resources used by the server cluster exceeds this capacity threshold, the platform will output an alarm message to the target user. If the user responds to the alarm message and instructs the platform to expand the server cluster, the platform will expand the server cluster. During the expansion process, to prevent the server cluster from crashing due to excessive resource usage, the platform will restrict the target user's write operations on the server cluster. This prevents the business services running on the server cluster from writing data, thus leading to a significant risk of business service failure.

[0057] In this disclosure, based on the user's server cluster resource usage data over a historical period, the resource growth rate of the user's current server cluster is predicted. Combined with the user's current server cluster's used and maximum resource capacity, the remaining available time of the user's current server cluster is calculated. When the remaining available time is insufficient, an alarm message is output to remind the user to expand the currently used server cluster in a timely manner. Since the user's current server cluster has relatively sufficient remaining available resources when the alarm message is output, the expansion operation of the server cluster can be supported without restricting user write operations. This can avoid the business failure risk caused by restricting user write operations and improve the robustness of the services running in the current server cluster.

[0058] It should be noted that the alarm information is only used to remind the target user. Based on the target user's historical resource usage analysis of the server cluster, the available time of the remaining resources in the first cluster is insufficient.

[0059] When the target user receives the alarm information, they can choose to respond to the alarm information and instruct the platform to expand the capacity of the first cluster to avoid the problem of insufficient available time for the current remaining resources of the first cluster; or they can choose not to respond to the alarm information.

[0060] In one embodiment, predicting the resource growth rate of the first cluster based on the plurality of resource quantity data includes:

[0061] The resource quantity data are analyzed based on a pre-acquired target regression model to obtain the resource quantity growth rate, wherein the target regression model is a neural network model.

[0062] In this embodiment, by applying a trained target regression model to analyze multiple resource quantity data, it can adapt to the ever-changing resource quantity usage in reality. This can improve the accuracy of the obtained resource quantity growth rate, thereby ensuring the reliability of the remaining available time calculated based on the resource quantity growth rate, and making the output of alarm information more timely and effective.

[0063] For example, the target regression model can be a Keras regression model.

[0064] In one embodiment, before predicting the resource growth rate of the first cluster based on the plurality of resource quantity data, the method further includes:

[0065] Multiple sets of training data are acquired, wherein each set of training data includes a first parameter and multiple second parameters. The first parameter indicates the growth rate of the amount of resources used by the corresponding cluster, and the second parameter indicates the amount of resources used by the corresponding cluster in a historical period. Different second parameters in each set of training data correspond to different historical periods.

[0066] The initial regression model is trained based on the multiple training data to obtain the target regression model.

[0067] For example, the second cluster may include the number of tables, the number of regions, the namespace capacity, and the namespace ID corresponding to the training user's use of the server cluster in historical time; wherein, the training user is one of multiple users who have accessed the platform and rented the server cluster, and the multiple sets of training data correspond to multiple different training users.

[0068] like Figure 2 As shown, the target regression model includes multiple hidden layers. The model input of the target regression model is a data point, which includes the number of tables, the number of regions, the namespace capacity, and the namespace ID. The target regression model converts the data point into a single neuron with a linear activation function. The single neuron indicates the growth rate of the amount of resources used by the corresponding server.

[0069] In one embodiment, before obtaining the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster, the method further includes:

[0070] Deploy the first sub-cluster for storing data, and the second sub-cluster for containerizing cluster components;

[0071] Build multiple component containers in the second sub-cluster;

[0072] The first cluster is generated based on the first sub-cluster and the multiple component containers.

[0073] In this embodiment, by using a container-based server cluster deployment method, the parameter configuration process during the expansion of the first cluster is eliminated. This can shorten the time consumed by the expansion operation of the first cluster, reduce the risk of server cluster downtime during expansion, and further improve the robustness of the services running in the first cluster.

[0074] For example, such as Figure 3 As shown, the deployment process for the first cluster can be as follows:

[0075] First, deploy a distributed storage cluster (which can be understood as the first sub-cluster mentioned above), that is, deploy an object storage or a fully distributed Hadoop cluster, specifically deployed on a physical machine outside the Kubernetes cluster, to provide data storage services, and shared by all HBase clusters in the platform;

[0076] Then redeploy the Kubernetes cluster (which can be understood as the aforementioned second sub-cluster) to orchestrate the HBase cluster service and containerize the HBase cluster service;

[0077] Finally, the HMaster, RegionServer, and Zookeeper components (which can be understood as containers of the aforementioned components) are deployed in containers through a Kubernetes cluster. Deploying in containers requires building images of the corresponding components, specifically including: HBase image and Zookeeper image; where HBase image is used to start HMaster service and RegionServer service, and Zookeeper image is used to start Zookeeper service.

[0078] When a target user applies to create a cluster, the platform will pre-allocate computing and storage resources, then create various cluster resources, followed by namespaces and network policies. After creation, the aforementioned cluster resources will reside in the Kubernetes cluster. Then, a Zookeeper cluster will be created. Once the Zookeeper cluster is functioning normally, HMaster and Regionserver services will be created synchronously, thus completing the creation of the first cluster.

[0079] In one embodiment, the historical information further includes multiple expansion waiting times, wherein the expansion waiting time is the difference between a first time and a second time, the second time is the generation time of the capacity warning information of the second cluster, and the first time is the generation time of the expansion information in response to the capacity warning information;

[0080] After obtaining the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster, the method further includes:

[0081] The average of the multiple expansion waiting times is determined as the target waiting time;

[0082] The alert time is determined based on the target waiting time and the preset expansion time, wherein the expansion time is the time consumed in expanding the first cluster.

[0083] In this embodiment, by statistically analyzing the average of multiple expansion waiting times of multiple second clusters within a historical time period, the degree of importance that the target user attaches to alarm information and the time consumed from receiving alarm information to finally issuing expansion instructions are predicted. Finally, by summarizing the target waiting time and the preset expansion time, the corresponding warning time for the target user is determined, which can adapt to each user's platform usage habits and output a more accurate warning time.

[0084] The determination of the alert time based on the target waiting time and the preset expansion time can be understood as: the sum of the target waiting time and the preset expansion time is determined as the alert time.

[0085] In applications, the scaling time can be determined by monitoring the number of Master cluster requests, the time to restart and split WAL, and the time consumed by the server to execute RPC, combined with the complexity and number of nodes of the user's scaling up of the cluster.

[0086] For multiple users accessing the platform, the target waiting time and the preset expansion time are different for each user.

[0087] The process for determining the capacity warning information of the second cluster can be as follows: the remaining available time of the resources of the second cluster in the corresponding historical period is lower than the warning time, or the amount of resources used by the second cluster in the corresponding historical period exceeds the preset resource threshold.

[0088] For example, if the generation time of the capacity warning information of the target user's second cluster is January 1st (which can be understood as the second time), and the generation time of the expansion information of the target user in response to the capacity warning information is January 3rd (which can be understood as the first time), then the corresponding expansion waiting time is 2 days.

[0089] In one embodiment, after outputting the alarm information, the method further includes:

[0090] Obtain the expansion command corresponding to the alarm information;

[0091] According to the expansion instruction, the first cluster is expanded to increase the maximum resource volume of the first cluster.

[0092] In this embodiment, when the target user responds to the alarm information and sends an expansion command to the platform, the platform responds to the expansion command and expands the first cluster corresponding to the target user to increase the maximum resource volume of the first cluster. After the expansion is completed, the remaining available time of the expanded first cluster is recalculated based on the maximum resource volume of the expanded first cluster, so as to realize dynamic early warning of the resource usage of the first cluster.

[0093] In one embodiment, the scaling up process for the first cluster according to the scaling up instruction includes:

[0094] According to the expansion instruction, the maximum computing resources of the first cluster are increased, and / or the maximum storage resources of the first cluster are increased.

[0095] In this embodiment, during the response to the expansion command, the maximum computing resources of the first cluster are adaptively increased, and / or the maximum storage resources of the first cluster are increased, in order to adapt to the ever-changing expansion needs of actual applications, while avoiding the disorderly expansion of the resources occupied by the first cluster and improving the resource utilization of the multiple computer resources controlled by the platform.

[0096] In this embodiment, the maximum resource amount may include the maximum computing resource amount of the first cluster and the maximum storage resource amount of the first cluster. Correspondingly, the resource amount difference may also include a first difference of the corresponding computing resource and a second difference of the corresponding storage resource. The resource amount growth rate may include a first rate of the corresponding computing resource and a second rate of the corresponding storage resource. When the remaining available time calculated based on the first difference and the first rate, and / or the remaining available time calculated based on the second difference and the second rate, is less than the warning time, the aforementioned alarm information is output.

[0097] For example, the computing resources include a central processing unit (CPU) and memory resources, and the storage resources include two types of resources: solid state disk (SSD) and hard disk drive (HDD).

[0098] See Figure 4 , Figure 4 This is a capacity management device provided in an embodiment of the present disclosure, such as... Figure 4 As shown, the capacity management device 400 includes:

[0099] The acquisition module 401 is used to acquire the amount of resources used by the first cluster, the maximum amount of resources of the first cluster, and the historical information of the corresponding second cluster. The first cluster and the second cluster are different server clusters corresponding to the same user. The activation time of the first cluster is later than the activation time of the second cluster. The historical information includes multiple resource data, which are used to indicate the amount of resources used by the second cluster in the corresponding historical period.

[0100] The prediction module 402 is used to predict the resource growth rate of the first cluster based on the multiple resource quantity data.

[0101] The determining module 403 is used to determine the remaining available time of the first cluster based on the amount of resources used, the maximum amount of resources, and the resource growth rate. The remaining available time is the ratio of the resource difference to the resource growth rate, and the resource difference is the difference between the maximum amount of resources and the amount of resources used.

[0102] The output module 404 is used to output alarm information when the remaining available time is less than the warning time, and the alarm information is used to remind the user to expand the capacity of the first cluster.

[0103] In one embodiment, the prediction module 402 is specifically used for:

[0104] The resource quantity data are analyzed based on a pre-acquired target regression model to obtain the resource quantity growth rate, wherein the target regression model is a neural network model.

[0105] In one embodiment, the device 400 further includes:

[0106] The training module is used to acquire multiple sets of training data and train the initial regression model based on the multiple sets of training data to obtain the target regression model. Each set of training data includes a first parameter and multiple second parameters. The first parameter indicates the growth rate of the amount of resources used by the corresponding cluster, and the second parameter indicates the amount of resources used by the corresponding cluster in a historical period. Different second parameters in each set of training data correspond to different historical periods.

[0107] In one embodiment, the device 400 further includes a cluster deployment module, which is specifically used for:

[0108] Deploy the first sub-cluster for storing data, and the second sub-cluster for containerizing cluster components;

[0109] Build multiple component containers in the second sub-cluster;

[0110] The first cluster is generated based on the first sub-cluster and the multiple component containers.

[0111] In one embodiment, the historical information further includes multiple expansion waiting times, wherein the expansion waiting time is the difference between a first time and a second time, the second time is the generation time of the capacity warning information of the second cluster, and the first time is the generation time of the expansion information in response to the capacity warning information;

[0112] The device 400 further includes:

[0113] The time calculation module is used to determine the average of the multiple expansion waiting times as the target waiting time, and to determine the warning time based on the target waiting time and the preset expansion time, wherein the expansion time is the time consumed to perform expansion processing on the first cluster.

[0114] In one embodiment, the device 400 further includes:

[0115] The expansion module is used to obtain the expansion instruction corresponding to the alarm information, and to perform expansion processing on the first cluster according to the expansion instruction, so as to increase the maximum resource volume of the first cluster.

[0116] In one embodiment, the expansion module is specifically used for:

[0117] According to the expansion instruction, the maximum computing resources of the first cluster are increased, and / or the maximum storage resources of the first cluster are increased.

[0118] The capacity management device 400 provided in this embodiment can implement the various processes in the above-described capacity management method embodiments, and will not be repeated here to avoid repetition.

[0119] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0120] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0121] like Figure 5As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0122] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0123] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as capacity management methods. For example, in some embodiments, the capacity management method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the capacity management method described above may be performed. Alternatively, in other embodiments, computing unit 501 may be configured to perform capacity management methods by any other suitable means (e.g., by means of firmware).

[0124] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0128] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0129] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0130] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A capacity management method, characterized in that, The method includes: The system obtains the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster. The first cluster and the second cluster are different server clusters corresponding to the same user. The activation time of the first cluster is later than that of the second cluster. The historical information includes multiple resource data, which are used to indicate the amount of resources used by the second cluster in the corresponding historical period. Based on the multiple resource quantity data, predict the resource quantity growth rate of the first cluster; The remaining available time of the first cluster is determined based on the amount of resources used, the maximum amount of resources, and the resource growth rate. The remaining available time is the ratio of the resource difference to the resource growth rate. The resource difference is the difference between the maximum amount of resources and the amount of resources used. If the remaining available time is less than the warning time, an alarm message is output to remind the user to expand the capacity of the first cluster.

2. The method according to claim 1, characterized in that, The step of predicting the resource growth rate of the first cluster based on the multiple resource quantity data includes: The resource quantity data are analyzed based on a pre-acquired target regression model to obtain the resource quantity growth rate, wherein the target regression model is a neural network model.

3. The method according to claim 2, characterized in that, Before predicting the resource growth rate of the first cluster based on the multiple resource quantity data, the method further includes: Multiple sets of training data are acquired, wherein each set of training data includes a first parameter and multiple second parameters. The first parameter indicates the growth rate of the amount of resources used by the corresponding cluster, and the second parameter indicates the amount of resources used by the corresponding cluster in a historical period. Different second parameters in each set of training data correspond to different historical periods. The initial regression model is trained based on the multiple training data to obtain the target regression model.

4. The method according to claim 1, characterized in that, Before obtaining the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster, the method further includes: Deploy the first sub-cluster for storing data, and the second sub-cluster for containerizing cluster components; Build multiple component containers in the second sub-cluster; The first cluster is generated based on the first sub-cluster and the multiple component containers.

5. The method according to claim 1, characterized in that, The historical information also includes multiple expansion waiting times, wherein the expansion waiting time is the difference between a first time and a second time, the second time is the generation time of the capacity warning information of the second cluster, and the first time is the generation time of the expansion information in response to the capacity warning information; After obtaining the used resources of the first cluster, the maximum resources of the first cluster, and the historical information of the corresponding second cluster, the method further includes: The average of the multiple expansion waiting times is determined as the target waiting time; The alert time is determined based on the target waiting time and the preset expansion time, wherein the expansion time is the time consumed in expanding the first cluster.

6. The method according to claim 1, characterized in that, After outputting the alarm information, the method further includes: Obtain the expansion command corresponding to the alarm information; According to the expansion instruction, the first cluster is expanded to increase the maximum resource volume of the first cluster.

7. The method according to claim 6, characterized in that, The process of scaling up the first cluster according to the scaling instruction includes: According to the expansion instruction, the maximum computing resources of the first cluster are increased, and / or the maximum storage resources of the first cluster are increased.

8. A capacity management device, characterized in that, The device includes: The acquisition module is used to acquire the amount of resources used by the first cluster, the maximum amount of resources of the first cluster, and the historical information of the corresponding second cluster. The first cluster and the second cluster are different server clusters corresponding to the same user. The activation time of the first cluster is later than the activation time of the second cluster. The historical information includes multiple resource data, which are used to indicate the amount of resources used by the second cluster in the corresponding historical period. The prediction module is used to predict the resource growth rate of the first cluster based on the multiple resource quantity data. The determination module is used to determine the remaining available time of the first cluster based on the amount of resources used, the maximum amount of resources, and the resource growth rate. The remaining available time is the ratio of the resource difference to the resource growth rate, and the resource difference is the difference between the maximum amount of resources and the amount of resources used. The output module is used to output alarm information when the remaining available time is less than the warning time, and the alarm information is used to remind the user to expand the capacity of the first cluster.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 7.

Images