System Capacity Prediction Method, Device, Electronic Device, and Storage Medium

Abstract

The present disclosure provides a system capacity prediction method, apparatus, electronic device, and storage medium, relating to the field of computer technologies, specifically to the fields of cloud services, artificial intelligence, and server technologies. The specific implementation solution is as follows: Obtain the capacity acquisition data of the system in a first time period, and classify it according to the application programs included in the system; Input the capacity acquisition data of the application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain the classification capacity prediction value corresponding to the application type; Determine the system capacity prediction value of the system in a second time period according to the classification capacity prediction values corresponding to each application type, where the first time period is before the second time period. The embodiments of the present disclosure can improve the prediction accuracy of the system capacity.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, specifically to the fields of cloud services, artificial intelligence, and server technologies, and particularly relates to a system capacity prediction method, apparatus, electronic device, and storage medium. Background Art

[0002] With the rapid development of Internet technologies, Internet applications have gradually become an important driving force for business innovation and enterprise quality improvement and efficiency increase. Therefore, it is crucial to ensure the service quality of Internet applications.

[0003] Performing capacity prediction on server resources can better improve the utilization effect of hardware resources, reasonably allocate hardware resources, and improve the service stability of servers. Summary of the Invention

[0004] The present disclosure provides a system capacity prediction method, apparatus, electronic device, and storage medium.

[0005] According to one aspect of the present disclosure, there is provided a system capacity prediction method, including:

[0006] Obtaining capacity acquisition data of a system in a first time period and classifying it according to application programs included in the system;

[0007] Inputting the capacity acquisition data of application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain a classification capacity prediction value corresponding to the application type;

[0008] Determining a system capacity prediction value of the system in a second time period according to the classification capacity prediction values corresponding to each application type, where the first time period is before the second time period.

[0009] According to one aspect of the present disclosure, there is provided a system capacity prediction apparatus, including:

[0010] An application classification module, configured to obtain capacity acquisition data of a system in a first time period and classify it according to application programs included in the system;

[0011] A classification prediction module, configured to input the capacity acquisition data of application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain a classification capacity prediction value corresponding to the application type;

[0012] A system prediction module, configured to determine a system capacity prediction value of the system in a second time period according to the classification capacity prediction values corresponding to each application type, where the first time period is before the second time period.

[0013] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the system capacity prediction method according to any embodiment of the present disclosure.

[0017] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the system capacity prediction method according to any embodiment of the present disclosure.

[0018] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, which when executed by a processor, implements the system capacity prediction method according to any embodiment of the present disclosure.

[0019] Embodiments of the present disclosure can improve the prediction accuracy of system capacity.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0022] Figure 1 is a schematic diagram of a system capacity prediction method provided according to an embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram of a system capacity prediction method provided according to an embodiment of the present disclosure;

[0024] Figure 3 is a schematic diagram of a system capacity prediction method provided according to an embodiment of the present disclosure;

[0025] Figure 4 is a schematic diagram of a system capacity prediction method provided according to an embodiment of the present disclosure;

[0026] Figure 5 is a scenario diagram of a system capacity prediction method provided according to an embodiment of the present disclosure;

[0027] Figure 6 is a scenario diagram of a system capacity prediction method provided according to an embodiment of the present disclosure;

[0028] Figure 7 It is a schematic diagram of a system capacity prediction device provided according to an embodiment of the present disclosure;

[0029] Figure 8 It is a block diagram of an electronic device for implementing the system capacity prediction method of the embodiment of the present disclosure. Specific Embodiments

[0030] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist in understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] Figure 1 It is a flowchart of a system capacity prediction method disclosed according to an embodiment of the present disclosure. This embodiment can be applied to the situation of predicting the capacity of a system in a second time period based on the capacity of the system in a first time period. The method of this embodiment can be executed by a system capacity prediction device, which can be implemented in a software and / or hardware manner and is specifically configured in an electronic device with certain data operation capabilities. The electronic device can be a client device or a server device, and the client device can be, for example, a mobile phone, a tablet computer, a vehicle-mounted terminal, and a desktop computer, etc.

[0032] S101, Obtain the capacity collection data of the system in the first time period and classify it according to the application programs included in the system.

[0033] The system may refer to a computer cloud service system. The system includes at least one application program (Application, app), and each of the application programs is configured on at least one computer server. The system includes computer servers that configure application programs, and the same application program can be configured on different computer servers. Correspondingly, the computer server and the application program have a many-to-many correspondence relationship.

[0034] The first time period may refer to a first time period before the current time, which is a historical time period. Among them, the first time period may include the current time. The capacity collection data is used to determine the resource usage status or load of the system. The capacity collection data may include at least one of the following: Central Processing Unit (CPU) consumption data, memory usage data, disk input / output data, network traffic data, and running status data, etc. Each data can be further refined. For example, the CPU consumption data may include the CPU utilization rate, usually expressed as a percentage. The running status data may include at least one of the following: response time, throughput, concurrency, and exception data, etc. The throughput can be expressed as the current throughput of the received traffic, Query Per Second (QPS). Optionally, the capacity collection data may include: the current throughput of the received traffic QPS and the resource consumption CPU usage.

[0035] Obtaining the capacity collection data of the system in the first time period may be obtaining the capacity collection data of each application program in the system in the first time period. Exemplarily, at least one container may be configured in a computer server. Different containers are used to configure independent running environments. The data included in an application program can be placed into a container, and the application program runs in this container. The capacity collection data of the application program may be data collection for the container in which the application program is deployed. In the first time period, the capacity collection data may be obtained periodically, and the capacity collection data may also be obtained randomly. What is obtained may be the capacity collection data of each application program, or may also be the capacity collection data of some application programs. Among them, the capacity collection data, the scope of the application programs collected, and the collection frequency may be limited as needed. In this regard, the embodiments of the present disclosure do not make specific limitations.

[0036] Classifying according to the application programs included in the system means classifying the capacity collection data of the system according to the types of application programs. The application programs may be classified in advance to obtain application programs of at least one application type, so as to obtain the application programs belonging to the same application type, and obtain the capacity collection data of the application programs belonging to the same application type.

[0037] S102, input the capacity collection data of the application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type, and obtain the classification capacity prediction value corresponding to the application type.

[0038] The application type refers to the type of application program. The classification capacity prediction model is used to predict future capacity data based on historical capacity data. Specifically, it is used to determine the classification capacity prediction value of the corresponding application type in the second time period according to the capacity collection data of the application program of the corresponding application type in the first time period. The application type corresponds to the classification capacity prediction model. For example, it can be a one-to-one correspondence. The classification capacity prediction value is used to predict the capacity data of the system. The classification capacity prediction value can be the capacity prediction value of all application programs belonging to the same application type in the second time period. The capacity prediction value can include at least one of the following: CPU consumption data, memory usage data, disk input / output data, network traffic data, and running status data, etc. The specific content included in the capacity prediction value can be further subdivided, and no specific limitation is imposed thereon. Optionally, the capacity prediction value includes the CPU usage.

[0039] For each application type, the classification capacity prediction model can be trained separately in advance. After the training is completed, for each application type, the capacity collection data of the application programs belonging to each application type is input into the corresponding pre-trained classification capacity prediction model to obtain the classification capacity prediction value corresponding to each application type. Among them, the training samples of the classification capacity prediction model corresponding to one application type can be: the capacity collection data of multiple application function programs of this application type in the third time period and the capacity collection data in the fourth time period, where the duration of the third time period is the same as the duration of the first time period, the duration of the fourth time period is the same as the duration of the second time period, and the duration between the third time period and the fourth time period is the same as the duration of the first time period and the duration of the second time period.

[0040] S103. According to the classification capacity prediction values corresponding to each application type, determine the system capacity prediction value of the system in the second time period, where the first time period is before the second time period.

[0041] The system capacity prediction value is used to determine the capacity data of the system in the second time period. The system capacity prediction value can determine the resource usage status or load of the system in a future time period, so that the capacity of the system can be adjusted according to the system capacity prediction value, and the quality and stability of the services provided by the system can be improved. The classification capacity prediction values corresponding to each application type are statistically processed to obtain the system capacity prediction value of the system in the second time period. The statistical method can be addition, weighting, multiplication, etc.

[0042] In the prior art, data is collected based on the overall capacity index of the system, and the system capacity is predicted. This method considers the entire system as a capacity issue and can only achieve relatively coarse-grained overall capacity prediction. It is impossible to accurately predict the system capacity data. In some cases, the total amount prediction under normal conditions can reach the system capacity credible data at the production environment level, but there are often large deviations in the accurate capacity scenario, resulting in overjudgment of the system capacity. For example, the rough accuracy leads to a large variance in capacity prediction, which causes the system to spend more resources to prepare for capacity expansion, while the utilization rate of the actual capacity is not high.

[0043] In addition, in the prior art, it is also possible to simulate pressure access to the traffic at the entrance of the complex system, so as to observe the overall system capacity load caused by the change of the overall entrance traffic of the system. According to the pressure and monitoring conditions of multiple modules in the system, it is used as a feedback signal of the system pressure, so as to obtain a system capacity data at the entrance level, which is used to measure the prediction relationship between the system entrance traffic and the system capacity. However, in a complex network topology, the computer service capacity of the cost backbone node will become the short board restricting the overall system capacity. After expanding the capacity of the short board service in the system, new short board services will be triggered. Identifying and expanding the service capacity through multiple rounds of pressure tests in a complex system will consume huge time and labor costs.

[0044] According to the technical solution of the present disclosure, by splitting the capacity collection data of the system in the first time period into capacity collection data corresponding to the application types and inputting them into the capacity prediction models corresponding to the application types, the classified capacity prediction values are predicted for each application type, and the system capacity prediction value of the system in the second time period is determined, so as to realize the splitting of the system capacity prediction into the capacity predictions of each application type. It is possible to accurately describe the capacity of an application program of an application type based on fine-grained decomposition of capacity prediction, improve the accuracy of capacity prediction, and effectively improve the accuracy of capacity prediction of a complex computer system, meeting the application requirements of capacity prediction for a large computer system.

[0045] Figure 2 It is a flowchart of another system capacity prediction method disclosed according to an embodiment of the present disclosure, which is further optimized and extended based on the above technical solution and can be combined with the above various optional implementation manners. The obtaining of the capacity collection data of the system in the first time period is specifically: obtaining the capacity collection data of the application programs included in the system in the first time period and forming a capacity collection data sequence of the application programs according to the collection time; determining the capacity collection data sequence of the application programs included in the system as the capacity collection data of the system in the first time period.

[0046] S201. Obtain the capacity acquisition data of the applications included in the system in the first time period, and form a capacity acquisition data sequence of the applications according to the acquisition time.

[0047] The capacity acquisition data sequence refers to the sequence of the capacity acquisition data of the same application arranged in the order of the acquisition time. At different times within the first time period, the capacity acquisition data of the application is obtained multiple times and arranged according to the acquisition time to form the capacity acquisition data sequence of the application. Exemplarily, when acquiring the capacity acquisition data of the i-th application at the t-th time, it can be represented by C i,t The capacity acquisition data of the i-th application in the first time period from the 1st time to the t-th time can be represented by C i,1:t represented as C i,1:t = [C 1,t , C 2,t , C 3,t , …… C i,t . Among them, at which time within the first time period to acquire the capacity acquisition data of the application can be configured as needed. Each time can be acquired randomly or according to rules, for example, periodically. Exemplarily, the times in the first time period are the 1st hour, the 2nd hour... the 23rd hour.

[0048] S202. Determine the capacity acquisition data sequence of the applications included in the system as the capacity acquisition data of the system in the first time period, and classify according to the applications included in the system.

[0049] Determine the capacity acquisition data sequences of all the applications in the system as the capacity acquisition data of the system in the first time period. Usually, the capacity acquisition data of the same parameter of different applications in the system will be accumulated to obtain the capacity acquisition data of this parameter of the system. In the embodiments of the present disclosure, the capacity acquisition data of different applications may not be accumulated and are distinguished according to the applications. Among them, the capacity acquisition data of the system includes the statistical values of the capacity data of all the applications at each time in the first time period. Exemplarily, it can be represented by C t represent the capacity acquisition data of the system at the t-th time, which is the accumulated value of the capacity acquisition data of n applications, that is Correspondingly, it can be represented by C 1:t represent the capacity acquisition data of the system from the 1st time to the t-th time, C 1:t = [C1, C2, …, C t .

[0050] The capacity collection data sequence of all included application programs is determined as the capacity collection data of the system in the first time period. The capacity collection data of the application programs are represented by sequences respectively and have a corresponding relationship with the application programs, so as to quickly classify the capacity collection data of the system according to the application programs.

[0051] Optionally, classifying according to the application programs included in the system includes: classifying the application programs included in the system according to classification parameters to obtain application programs of at least one application type, where the classification parameters include at least one of the following: function type parameter, running time parameter, and online / offline parameter; obtaining the capacity collection data of the application programs of each application type.

[0052] The classification parameters serve as the basis for classifying application programs and are used to classify application programs. The function parameter is used to classify according to the function (or service) of the application program. Exemplarily, near-line application programs, streaming application programs, or caching application programs, etc. can be classified; the running time parameter is used to classify according to the commonly used running time of the application program. Exemplarily, application programs running during the day, application programs running at night, application programs running on weekdays, or application programs running on holidays, etc. can be classified; the online / offline parameter is used to classify according to whether the application program is running online or offline. Exemplarily, offline traffic application programs or online traffic application programs, etc. can be classified. In addition, multiple classification parameters can be mixed for classification. Exemplarily, the application programs can be classified according to the joint cooperation of the function parameter and the online / offline parameter. Exemplarily, offline batch calculation application programs or online caching application programs, etc. can be classified. In addition, there are other contents for the classification parameters, and they can be combined according to needs, which are not limited herein.

[0053] Classifying the capacity collection data according to the application programs included in the system is actually classifying the application programs. The capacity collection data of the application programs belonging to the same application type corresponds to the application type. Thus, the capacity collection data can be grouped according to the application type, and the capacity collection data with the same characteristics can be put together to predict the classification capacity prediction value, which can improve the prediction accuracy of the classification capacity prediction value.

[0054] By configuring the classification parameters into multiple contents, the diversity of classification and the fine granularity of classification are increased, so as to realize that the capacity collection data of the application programs of the same application type are used as the same group of data for prediction, which can enrich the characteristics of the application type to which the capacity collection data in this group belongs, and then perform classification capacity prediction, which can improve the accuracy of classification capacity prediction.

[0055] S203. Input the capacity collection data of application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain the classification capacity prediction value corresponding to the application type.

[0056] S204. Determine the system capacity prediction value of the system in the second time period based on the classification capacity prediction values corresponding to each application type, where the first time period is before the second time period.

[0057] Exemplarily, the first time period is from the 1st time to the tth time, and the second time period is from the (t + 1)th time to the (t + a)th time after the tth time. Optionally, the second time period is a time period of a duration after the first time period, that is, p is t + 1 and q is t + a. Correspondingly, the system capacity prediction value can be expressed as the capacity data from the (t + 1)th time to the (t + a)th time after the tth time. According to C 1:t Solve for C 1+t:t+a value.

[0058] Specifically, the system capacity prediction value can construct an overall formal model where C <x is the capacity value of the overall capacity in the time series [1, x).

[0059] Dividing n application programs into M groups can form a capacity collection data sequence corresponding to the medium-grained application type where y ∈ {1, …, M} is a type variable, is the capacity collection data of the y application type at the tth time, which is obtained by aggregating the capacity collection data of the application programs belonging to the y application type.

[0060] Secondly, perform separate modeling on the medium-grained aggregated capacity time series using a mature machine learning training and prediction scheme to obtain the prediction result Summarize to obtain a sequence representing the system capacity prediction value

[0061] Finally, use the machine learning mixture probability distribution, and the mixture model is generated as described below

[0062]

[0063] Among them, y is an offline latent variable, M is the number of components of the mixture model, that is, the number of application types, p(y) is the prior distribution of the sample category, p(c|y) is the cost generation probability corresponding to the y-th application type for the fine-grained time series c. Solve for the modeling ability. Each medium-grained grouped time series comes from a neural network model determined by different parameters. Among them, the capacity acquisition data of the applications of each application type can be divided into the group of that application type, so as to form multiple classification capacity prediction models with different parameters. Finally, the parameters of the mixture model are θ = {Φ1, Φ2, …, Φ M}.

[0064] Optionally, determining the system capacity prediction value of the system in the second time period according to the classification capacity prediction values corresponding to each application type includes: adding up the classification capacity prediction values corresponding to each application type to obtain the system capacity prediction value of the system in the second time period.

[0065] Usually, the capacity data of the system is the data obtained by adding up the capacity data of each application included in the system. Correspondingly, the sum of the classification capacity prediction values of different application types is the system capacity prediction value. Exemplarily, Y t represents the system capacity prediction value of the system at the t-th time, and Y i,t is the classification capacity prediction value of the i-th application type.

[0066] By adding up the classification capacity prediction values to obtain the system capacity prediction value, the acquisition method of the system capacity prediction value can be simplified, and at the same time, the prediction speed of the system capacity can be accelerated, and the system capacity prediction efficiency can be improved.

[0067] Optionally, the capacity acquisition data includes: throughput and / or resource consumption; the system capacity prediction value includes: resource consumption.

[0068] Among them, the throughput can refer to the current traffic QPS, and the resource consumption can refer to the CPU usage. QPS affects the CPU usage, so QPS can be used to measure the CPU usage, and furthermore, QPS can be used to predict the CPU usage. The resource consumption can be used to characterize the performance, pressure resistance, and load of the system, etc.

[0069] In addition, the capacity prediction value can also include the resource utilization rate. The total CPU of the applications included in the system can be calculated to obtain the total CPU of the system. The ratio of the CPU usage included in the system capacity prediction value to the total CPU of the system is determined as the predicted CPU utilization rate, and the CPU utilization rate, that is, the resource utilization rate, can also be determined as the system capacity prediction value.

[0070] By configuring the specific content of the capacity acquisition data and the predicted value of the system capacity, it is possible to evaluate the performance, load, and compression resistance of the system, etc., so as to expand or contract the application programs and computer servers of the system, effectively utilize the system capacity, reduce the waste of the system capacity, and promptly respond to the situation of insufficient capacity, thereby improving the stability of the system.

[0071] According to the technical solution of the present disclosure, by obtaining the capacity acquisition data of the application program in the first time period, forming a sequence of capacity acquisition data of the application program according to the acquisition time, and determining the sequence of capacity acquisition data of all application programs in the system as the capacity acquisition data of the system in the first time period, it is possible to obtain the capacity acquisition data of each application program in the system, increase the capacity acquisition range of the system, and perform capacity prediction for different application types to achieve accurate prediction of the system capacity. At the same time, by obtaining the capacity acquisition data at multiple acquisition times in the first time period, it is possible to obtain the relationship between the capacity data and time of the application program, increase the characteristics of capacity and time, so as to predict the capacity data in the future time period and improve the accuracy of capacity prediction in the second time period.

[0072] Figure 3 It is a flowchart of another system capacity prediction method disclosed according to an embodiment of the present disclosure, which is further optimized and extended based on the above technical solution and can be combined with each of the above optional implementation manners. The classification capacity prediction model corresponding to the application type is specifically: the application capacity prediction model corresponding to the application program; the step of inputting the capacity acquisition data of the application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain the classification capacity prediction value corresponding to the application type is specifically: among the application programs belonging to the same application type, inputting the capacity acquisition data of the application program into the application capacity prediction model corresponding to the application program to obtain the application capacity prediction value of the application program; and adding up the application capacity prediction values of the application programs belonging to the same application type to obtain the classification capacity prediction value corresponding to the application type.

[0073] S301, obtain the capacity acquisition data of the system in the first time period and classify it according to the application programs included in the system.

[0074] S302, among the application programs belonging to the same application type, input the capacity acquisition data of the application program into the application capacity prediction model corresponding to the application program to obtain the application capacity prediction value of the application program.

[0075] The application capacity prediction model corresponding to the application is used to predict the application capacity prediction value of the application in the second time period according to the capacity acquisition data of the application in the first time period. The application capacity prediction value is the capacity data of the application in the second time period. In fact, in the embodiments of the present disclosure, the classification capacity prediction model corresponding to the application type is refined into the application capacity prediction model corresponding to each application of the application type. That is, the classification capacity prediction of an application type is refined into the application capacity prediction of each application under the application type. Among them, the training samples of the application capacity prediction model corresponding to an application can be: the capacity acquisition data of the application in the third time period and the capacity acquisition data in the fourth time period, where the duration of the third time period is the same as the duration of the first time period, the duration of the fourth time period is the same as the duration of the second time period, and the duration between the third time period and the fourth time period is the same as the duration of the first time period and the duration of the second time period.

[0076] The application capacity prediction models corresponding to the applications of the same application type can be the same. The application capacity prediction models corresponding to the applications of different application types can be different. In the same case, the training samples of the application capacity prediction model corresponding to an application can be: the capacity acquisition data of multiple applications of the same model in the third time period and the capacity acquisition data in the fourth time period.

[0077] S303. Accumulate the application capacity prediction values of the applications belonging to the same application type to obtain the classification capacity prediction value corresponding to the application type.

[0078] The classification capacity prediction value of an application type is the accumulation of the application capacity prediction values of all applications under the application type.

[0079] S304. Determine the system capacity prediction value of the system in the second time period according to the classification capacity prediction values corresponding to the application types, where the first time period is before the second time period.

[0080] Optionally, the application capacity prediction model corresponding to the application includes at least one duration capacity prediction model of a duration type; the step of inputting the capacity acquisition data of the application into the application capacity prediction model corresponding to the application to obtain the application capacity prediction value of the application includes: inputting the capacity acquisition data of the application into each duration capacity prediction model of the duration type corresponding to the application to obtain the duration capacity prediction values corresponding to the duration types; obtaining the weights of the duration types; and performing weighted calculation on the duration capacity prediction values corresponding to the duration types according to the weights of the duration types to obtain the application capacity prediction value of the application.

[0081] The duration type is used to classify the duration of the first time period and the capacity collection data of the first time period. The duration type can refer to the type of the duration of the first time period. Exemplarily, the duration type can include one day, n days, one week, n weeks, one month, n months, one year, or n years, etc. The capacity collection data of the first time period with different duration types can have different characteristics, so that they can be predicted independently, further improving the accuracy of capacity prediction in terms of time. The duration capacity prediction model is used to predict the duration capacity prediction value of the application program in the second time period according to the capacity collection data corresponding to the duration type by the application program. The duration capacity prediction value is the capacity data predicted by the application program in the second time period based on the capacity collection data corresponding to the duration type. The duration capacity prediction value is used to combine the duration capacity prediction values of other duration types to determine the application capacity prediction value. Among them, the training sample of a duration capacity prediction model corresponding to a duration type can be: the capacity collection data of the application program corresponding to the duration type and the capacity collection data in the fourth time period, where the third time period includes the time period covered by the collection time of the capacity collection data corresponding to the duration type, the duration of the third time period is the same as the duration of the first time period, the duration of the fourth time period is the same as the duration of the second time period, and the duration between the third time period and the fourth time period is the same as the duration of the first time period and the duration of the second time period.

[0082] Input the capacity collection data of the application program into the duration capacity prediction models corresponding to each of the duration types of the application program to obtain the duration capacity prediction values corresponding to each duration type. It can be: in the capacity collection data of the application program, obtain the capacity collection data corresponding to the duration type and input it into the duration capacity prediction model of this duration type to obtain the duration capacity prediction value corresponding to this duration type.

[0083] The weights of different duration types can be different or the same. The weighted calculation can be: multiply the weight of the duration type by the duration capacity prediction value corresponding to this duration type, and accumulate the products of each duration type to obtain the application capacity prediction value of the application program.

[0084] Among them, the weights can be configured according to business requirements or the application scenarios. Exemplarily, the application scenarios during the day and / or at night can increase the weights of the one-day type, and the application scenarios on weekdays and / or weekends can increase the weights of the one-week type. In a specific example, the capacity collection data obtained in the first time period includes the capacity collection data of each application program on Monday of a certain week and the capacity collection data of the previous week, and the second time period is one day of Tuesday. For each application program, the capacity collection data on Monday is input into the corresponding one-day type duration capacity prediction model to obtain the first result A, and the capacity collection data of the previous week is input into the corresponding one-week type duration capacity prediction model to obtain the second result B. Among them, the weight of the one-day type is 0.7, and the weight of the one-week type is 0.3. The application capacity prediction value of this application program is 0.7*A + 0.3*B.

[0085] By refining the capacity collection data corresponding to the application type into the capacity collection data under different duration types and inputting them into the capacity prediction model corresponding to the duration type, the duration capacity prediction value is predicted for each duration type, and weighted calculation is performed according to the weights of the duration types to determine the application capacity prediction value of the application program in the second time period, so as to realize the splitting of the application capacity prediction into the capacity predictions of different durations, further refine and decompose the capacity prediction, accurately depict the capacity of the application program at different times, improve the accuracy of capacity prediction, and at the same time, the weights can be adapted to different duration application scenarios, enrich the application scenarios of capacity prediction, and thus improve the accuracy of capacity prediction.

[0086] According to the technical solution of the present disclosure, by refining the capacity collection data corresponding to the application type into the capacity collection data of each application program under this application type and inputting them into the capacity prediction model corresponding to the application program, the application capacity prediction value is predicted for each application program, and the classification capacity prediction value of this application type in the second time period is determined, so as to realize the splitting of the classification capacity prediction into the capacity predictions of each application program, and based on the fine-grained decomposition of the capacity prediction, accurately depict the capacity of an application program, improve the accuracy of capacity prediction, and effectively improve the capacity prediction accuracy of complex computer systems, meeting the application requirements of capacity prediction for large computer systems.

[0087] Figure 4 It is a scenario diagram of another system capacity prediction method disclosed according to an embodiment of the present disclosure. The system capacity prediction method may include:

[0088] S401, obtain the overall collection capacity of the complex system.

[0089] The overall collection capacity is the capacity collection data of the application programs included in the system, specifically including throughput and / or resource consumption. Obtain the capacity collection data of the system in the first time period and classify it according to the application programs included in the system.

[0090] Specifically, the system collects capacity data C from the first time to the t-th time 1:t =[C1, C2, …, C t . The capacity data collected by the system at the t-th time Among them, the capacity data collected by the i-th application at the t-th time is C i,t . And the capacity data collected by the i-th application in the first time period from the first time to the t-th time is C i,1:t =[C 1,t , C 2,t , C 3,t , …… C i,t .

[0091] S402. Split into M grouping types according to the application type for prediction.

[0092] Input the capacity data of the applications belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain the classification capacity prediction value corresponding to the application type.

[0093] Specifically, the capacity data sequence corresponding to the application type Among them, y ∈ {1, …, M} is a type variable, is the capacity data collected by the y application type at the t-th time, which is aggregated from the capacity data of the applications belonging to the y application type. The classification capacity prediction value corresponding to the application type is

[0094] S403. Aggregate the M medium-grained classification capacity prediction values into an overall system capacity prediction value.

[0095] According to the classification capacity prediction values corresponding to each application type, determine the system capacity prediction value of the system in the second time period. The first time period is before the second time period. The capacity prediction value is the resource consumption.

[0096] Specifically, add up the classification capacity prediction values corresponding to each application type to obtain

[0097] In a specific example, such as Figure 5As shown, obtain the capacity collection data of fine-grained application programs, group them according to the application types of the application programs, construct the topological structure of the mesh, and obtain the capacity collection data of medium-grained application types. Among them, the application types can be determined by different dimensions (classification parameters) to achieve multi-dimensional capacity prediction. For each application type, the capacity collection data of this application type is used to predict the classification capacity prediction value corresponding to this application type. Accumulate the classification capacity prediction values of multiple application types to obtain the overall system capacity of the complex system and pre-store it.

[0098] For the capacity prediction splitting method of application classification, by introducing more types of other-dimensional time series, such as night and day dimensions, the types of fine-grained application function programs can be divided more finely and accurately, thereby increasing the number of time series groups, making the described dimensions more detailed, and the aggregated multiple classification capacity prediction values can more accurately predict the overall predicted capacity of the system.

[0099] Furthermore, the classification capacity prediction model corresponding to the application type includes: the capacity collection data corresponding to at least one duration type of the application program, specifically including: the duration capacity prediction model for one day and the duration capacity prediction model for one week; in the capacity collection data of the application program, obtain the capacity collection data corresponding to the duration type and input it into the duration capacity prediction model of this duration type to obtain the duration capacity prediction value corresponding to this duration type. Obtain the weights of each duration type; according to the weights of each duration type, perform weighted calculation on the duration capacity prediction values corresponding to each duration type to obtain the application capacity prediction value of this application program. Accumulate the application capacity prediction values of the application programs belonging to the same application type to obtain the classification capacity prediction value corresponding to the application type.

[0100] As Figure 6 shown, multiple computer service application programs (Application) in an online computer system will perform network connection and topological organization according to the business scenario. Each computer service application program is composed of multiple service instances (Instance), and each instance will generate a series of data metrics related to capacity at different time periods. The metrics related to capacity are usually the QPS of the service instance and the CPU utilization rate cpu_percent of the service instance. Collect the QPS and CPU utilization rate produced by the service instance in multiple time periods, and persistently store these collected data to prepare for the next step of processing.

[0101] Aggregate and filter the QPS and CPU utilization of service instances to obtain the capacity collection data corresponding to each duration type. Specifically, first summarize by service instance, then aggregate by computer service application, filter out some abnormal fluctuation data through the set variance threshold, and finally map the QPS and CPU utilization to the time series to form the capacity collection data, and determine the capacity collection data corresponding to the one-day duration type and the capacity collection data corresponding to the one-week duration type according to the duration type.

[0102] Build duration capacity prediction models for the one-day dimension and the one-week dimension, construct a training model for the existing data samples, so as to estimate the CPU usage of the application. Among them, the duration capacity prediction model of the one-day duration type corresponds to the duration capacity prediction model of the one-day dimension, and the duration capacity prediction model of the one-week duration type corresponds to the duration capacity prediction model of the one-week dimension. Building duration capacity prediction models for the one-day dimension and the one-week dimension for the same computer program service application is to meet the capacity fluctuations in different scenarios, and at the same time, cross-validation and use of the two models can be carried out. For example, the duration capacity prediction model of the one-day dimension is used in scenarios during the day and / or at night, and the duration capacity prediction model of the one-week dimension is used in scenarios that distinguish weekdays and / or weekends. According to different application scenarios, different weights are configured for different duration types, and different duration capacity prediction values are weighted to obtain an application capacity prediction value for an application. Thus, the application capacity prediction value corresponding to each application can be obtained.

[0103] Divide computer service applications of the same application type, and the application types include: online service, offline service, offline batch computing service, streaming service or nearline service, etc. Through the aggregation of the same applications, a medium-grained classification capacity prediction model can be formed, and the classification capacity prediction model of the application type can be accurately deduced and predicted through the duration capacity prediction models of the one-day dimension and the one-week dimension. And through the aggregation of the application capacity prediction values of the same application type, a medium-grained classification capacity prediction value is formed. Through multiple classification capacity prediction values, the overall capacity data of the complex system, that is, the system capacity prediction value, can be predicted.

[0104] An embodiment of the present disclosure proposes a grouping capacity prediction method based on medium-grained time series splitting. For fine-grained computer service applications, collection and time-sharing modeling are adopted for implementation. The capacity is predicted from fine-grained time series applications and aggregated to medium-grained grouped time series classification for capacity prediction. Finally, the system prediction capacity of the entire complex system is realized, improving the accuracy and precision of the overall capacity estimation. Currently, it can be applied in search recommendation systems, information flow recommendation systems, and App cloud service systems, covering hundreds of thousands of computer service applications and tens of thousands of machines in the online system.

[0105] According to an embodiment of the present disclosure, Figure 7 is a structural diagram of a system capacity prediction device in an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the situation of predicting the capacity of a system in a second time period based on the capacity of the system in a first time period. The device is implemented by software and / or hardware and is specifically configured in an electronic device with certain data operation capabilities.

[0106] As Figure 7 shown, a system capacity prediction device 700 includes: an application classification module 701, a classification prediction module 702, and a system prediction module 703; wherein,

[0107] The application classification module 701 is configured to obtain the capacity collection data of the system in the first time period and classify it according to the application programs included in the system.

[0108] The classification prediction module 702 is configured to input the capacity collection data of the application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain the classification capacity prediction value corresponding to the application type.

[0109] The system prediction module 703 is configured to determine the system capacity prediction value of the system in the second time period according to the classification capacity prediction values corresponding to each application type, where the first time period is before the second time period.

[0110] According to the technical solution of the present disclosure, by splitting the capacity collection data of the system in the first time period into capacity collection data corresponding to application types and inputting it into the capacity prediction model corresponding to the application type, the classification capacity prediction value is predicted for each application type, and the system capacity prediction value of the system in the second time period is determined, realizing the splitting of the system capacity prediction into the capacity predictions of each application type. It can accurately depict the capacity of the application programs of an application type based on fine-grained decomposition of capacity prediction, improve the accuracy rate of capacity prediction, and effectively improve the capacity prediction accuracy rate of complex computer systems, meeting the application requirements of large-scale computer system capacity prediction.

[0111] Further, the application classification module 701 includes: a time series acquisition unit, configured to acquire capacity acquisition data of application programs included in the system in a first time period, and form a capacity acquisition data sequence of the application programs according to the acquisition time; an application sequence statistics unit, configured to determine the capacity acquisition data sequence of the application programs included in the system as the capacity acquisition data of the system in the first time period.

[0112] Further, the application classification module 701 includes: an application program division unit, configured to classify the application programs included in the system according to classification parameters to obtain application programs of at least one application type, where the classification parameters include at least one of the following: a function type parameter, a running time parameter, and an online / offline parameter; a capacity acquisition data classification and statistics unit, configured to acquire capacity acquisition data of the application programs of each application type.

[0113] Further, the classification capacity prediction model corresponding to the application type includes: an application capacity prediction model corresponding to an application program; the classification prediction module 702 includes: an application capacity prediction unit, and a vector coordinate point acquisition unit, configured to input the capacity acquisition data of the application program into the application capacity prediction model corresponding to the application program among the application programs belonging to the same application type to obtain an application capacity prediction value of the application program; an application capacity accumulation unit, configured to accumulate the application capacity prediction values of the application programs belonging to the same application type to obtain a classification capacity prediction value corresponding to the application type.

[0114] Further, the application capacity prediction model corresponding to the application program includes at least one duration type duration capacity prediction model; the application capacity prediction unit includes: a duration dimension prediction subunit, configured to input the capacity acquisition data of the application program into each of the duration type duration capacity prediction models corresponding to the application program to obtain duration capacity prediction values corresponding to each duration type; a duration weight acquisition subunit, configured to acquire weights of each duration type; a duration prediction weighting subunit, configured to perform weighted calculation on the duration capacity prediction values corresponding to each duration type according to the weights of each duration type to obtain an application capacity prediction value of the application program.

[0115] Further, the application classification module 701 includes: a classification capacity accumulation unit, configured to accumulate the classification capacity prediction values corresponding to each application type to obtain a system capacity prediction value of the system in a second time period.

[0116] Further, the capacity acquisition data includes: throughput and / or resource consumption; the system capacity prediction value includes: resource consumption.

[0117] The above-mentioned system capacity prediction device can execute the system capacity prediction method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the system capacity prediction method.

[0118] In the technical solution of the present disclosure, the processing of collection, storage, use, processing, transmission, provision, and disclosure of user personal information complies with the provisions of relevant laws and regulations and does not violate public order and good customs.

[0119] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0120] Figure 8 A schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0121] As Figure 8 shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0122] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0123] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the system capacity prediction method. For example, in some embodiments, the system capacity prediction method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the system capacity prediction method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the system capacity prediction method by any other suitable means (e.g., by means of firmware).

[0124] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0128] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0129] A computer system can include a client and a server. The client and the server are generally far apart from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, can also be a server of a distributed system, or a server incorporating a blockchain.

[0130] It should be understood that the various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0131] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A system capacity prediction method, comprising: Obtaining capacity collection data of the system in a first time period and classifying it according to the application programs included in the system; Inputting the capacity collection data of the application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain a classification capacity prediction value corresponding to the application type; wherein, the classification capacity prediction model corresponding to the application type includes an application capacity prediction model corresponding to the application program; the application capacity prediction model corresponding to the application program includes at least one duration capacity prediction model of a duration type; wherein, different durations correspond to different duration types; Determining a system capacity prediction value of the system in a second time period according to the classification capacity prediction values corresponding to each application type, the first time period being before the second time period; The step of inputting the capacity collection data of the application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type to obtain a classification capacity prediction value corresponding to the application type includes: Among the application programs belonging to the same application type, inputting the capacity collection data of the application program into the duration capacity prediction models of each duration type corresponding to the application program to obtain duration capacity prediction values corresponding to each duration type; Obtaining the weights of each duration type; Performing weighted calculation on the duration capacity prediction values corresponding to each duration type according to the weights of each duration type to obtain an application capacity prediction value of the application program; Accumulating the application capacity prediction values of the application programs belonging to the same application type to obtain a classification capacity prediction value corresponding to the application type.

2. The method according to claim 1, wherein, The step of obtaining capacity collection data of the system in the first time period includes: Obtaining capacity collection data of the application programs included in the system in the first time period and forming a capacity collection data sequence of the application programs according to the collection time; Determining the capacity collection data sequence of the application programs included in the system as the capacity collection data of the system in the first time period.

3. The method according to claim 1, wherein The step of classifying according to the application programs included in the system includes: Classifying the application programs included in the system according to classification parameters to obtain application programs of at least one application type, the classification parameters including at least one of the following: function type parameter, running time parameter, and online / offline parameter; Obtaining capacity collection data of the application programs of each application type.

4. The method according to claim 1, wherein, The step of determining a system capacity prediction value of the system in the second time period according to the classification capacity prediction values corresponding to each application type includes: Accumulating the classification capacity prediction values corresponding to each application type to obtain a system capacity prediction value of the system in the second time period.

5. The method according to claim 1, wherein, The capacity collection data includes: throughput and / or resource consumption; the system capacity prediction value includes: resource consumption.

6. A system capacity prediction device, comprising: An application classification module, configured to obtain capacity collection data of the system in a first time period and classify it according to the application programs included in the system; A classification prediction module, configured to input the capacity acquisition data of application programs belonging to the same application type into the classification capacity prediction model corresponding to the application type, so as to obtain the classification capacity prediction value corresponding to the application type; A system prediction module, configured to determine the system capacity prediction value of the system in the second time period according to the classification capacity prediction values corresponding to each application type, where the first time period is before the second time period; Wherein, the classification capacity prediction model corresponding to the application type includes: an application capacity prediction model corresponding to the application program; The classification prediction module includes: An application capacity prediction unit, and a vector coordinate point acquisition unit, configured to input the capacity acquisition data of the application program into the application capacity prediction model corresponding to the application program among the application programs belonging to the same application type, so as to obtain the application capacity prediction value of the application program; An application capacity accumulation unit, configured to accumulate the application capacity prediction values of the application programs belonging to the same application type to obtain the classification capacity prediction value corresponding to the application type; Wherein, the application capacity prediction model corresponding to the application program includes at least one duration capacity prediction model of a duration type; The application capacity prediction unit includes: A duration dimension prediction subunit, configured to input the capacity acquisition data of the application program into each duration capacity prediction model of the duration type corresponding to the application program, so as to obtain the duration capacity prediction values corresponding to each duration type; A duration weight acquisition subunit, configured to acquire the weights of each duration type; A duration prediction weighting subunit, configured to perform weighted calculation on the duration capacity prediction values corresponding to each duration type according to the weights of each duration type, so as to obtain the application capacity prediction value of the application program; Wherein, the duration type corresponds to different durations.

7. The device according to claim 6, wherein The application classification module includes: A time series acquisition unit, configured to acquire the capacity acquisition data of the application programs included in the system in the first time period, and form a capacity acquisition data sequence of the application programs according to the acquisition time; An application sequence statistics unit, configured to determine the capacity acquisition data sequence of the application programs included in the system as the capacity acquisition data of the system in the first time period.

8. The apparatus according to claim 6, wherein The application classification module includes: An application program partitioning unit, configured to classify the application programs included in the system according to classification parameters to obtain application programs of at least one application type, where the classification parameters include at least one of the following: a function type parameter, a running time parameter, and an online / offline parameter; A capacity acquisition data classification statistics unit, configured to acquire the capacity acquisition data of the application programs of each application type.

9. The device according to claim 6, wherein The application classification module includes: A classification capacity accumulation unit, configured to accumulate the classification capacity prediction values corresponding to each application type to obtain the system capacity prediction value of the system in the second time period.

10. The apparatus according to claim 6, wherein The capacity acquisition data includes: throughput and / or resource consumption; the system capacity prediction value includes: resource consumption.

11. An electronic device, including: At least one processor; And A memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the system capacity prediction method according to any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are for causing the computer to execute the system capacity prediction method according to any one of claims 1-5.

13. A computer program product, comprising a computer program which, when executed by a processor, implements the system capacity prediction method according to any one of claims 1-5.

Images