Capacity prediction method, system, storage medium and processor

Abstract

The embodiment of the application provides a capacity prediction method, system, processor and storage medium, and belongs to the technical field of computers. The method comprises the following steps: collecting index data of a to-be-predicted capacity, and preprocessing the index data to obtain preprocessed index data; wherein the index data comprises index historical data, index real-time data, business data, associated resource data and application performance data; performing basic data classification based on the preprocessed index data; performing capacity prediction of a future preset time section in a capacity prediction model based on the classified basic data, to obtain a prediction result; and determining and pushing a prediction value of each capacity index based on the prediction result. The application solves the problem of low prediction accuracy of the existing capacity prediction scheme.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a capacity prediction method, a capacity prediction system, a storage medium, and a processor. Background Technology

[0002] To ensure stable system operation, it is essential to guarantee that system capacity meets demand. With the diversification of business types, the non-linear characteristics of capacity prediction are becoming increasingly apparent. Ensuring accurate capacity prediction and implementing early warnings, capacity cleanup, and expansion based on the prediction results can significantly guarantee stable system performance. For capacity warnings, existing operations teams typically rely on monitoring tools such as Zabbix, configuring monitoring items and alarm triggers for user-focused metrics. They monitor relevant capacity usage in real time and passively expand capacity based on their own operational experience. Secondly, based on the historical operational experience of operations personnel, supervisors assess capacity expansion trends and manage capacity accordingly. Additionally, machine learning methods can be used to predict capacity over a future period, and capacity management can be implemented based on future capacity trends.

[0003] Current capacity forecasting methods mostly use regression prediction based on the capacity index to be predicted. These regression models are generally linear, making it difficult to represent highly complex data and susceptible to outliers. In the modeling process, current capacity forecasting methods typically use historical data of the capacity index to predict future capacity. This approach fails to consider interrelated factors such as business operations, upstream and downstream data, and network topology configuration data, resulting in a limited data set and an inability to effectively predict capacity changes caused by business variations, changes in related resources, and changes in upstream and downstream businesses. Furthermore, regression prediction methods do not incorporate time characteristics into their modeling. As a typical application of time series analysis, time characteristics are a crucial factor influencing capacity trends. Therefore, to address the insufficient prediction accuracy of existing capacity forecasting schemes, a new capacity forecasting method is needed. Summary of the Invention

[0004] The purpose of this application is to provide a capacity prediction method, system, storage medium, and processor to solve the problem of low capacity prediction accuracy in existing solutions.

[0005] To achieve the above objectives, the first aspect of this application provides a capacity prediction method, comprising: collecting indicator data of the capacity to be predicted, and preprocessing the indicator data to obtain preprocessed indicator data; wherein the indicator data includes: historical indicator data, real-time indicator data, business data, associated resource data, and application performance data; classifying basic data based on the preprocessed indicator data; predicting the capacity for a future preset time period in a capacity prediction model based on the classified basic data, and obtaining prediction results; and determining and pushing the predicted values ​​of each capacity indicator based on the prediction results.

[0006] In this embodiment of the application, the associated resource data includes one or more of the following: total number of CPU cores, allocated CPU resources, remaining allocable CPU resources, total memory, allocated memory resources, and remaining usable memory resources.

[0007] In this embodiment of the application, the preprocessing of the indicator data includes: performing one or more of the following processes on the indicator data: outlier removal, deduplication, missing segment repair, and data removal; identifying time-series data, and extracting future covariates from the identified time-series data.

[0008] In this embodiment of the application, the method further includes: training a capacity prediction model, comprising: splitting historical data of the indicator into training data and validation data, wherein the training data is earlier than the validation data in time series; based on the training data, reading the corresponding concurrent auxiliary prediction time series, known future covariates, and static covariates, and forming a training set together with the training data; in a pre-constructed neural network, using the concurrent data of the validation data as the training target, training the model based on the training set to obtain an initial model; comparing the training result of the initial model with the validation data, and correcting the model based on the deviation, wherein after each model correction, the training result of the corrected model is compared with the validation data again until the difference between the two meets the error minimization condition, stopping the model correction, and using the corrected model obtained from the latest correction as the capacity prediction model.

[0009] In this embodiment of the application, the pre-built neural network includes: an input module, an encoder, a decoder, and an output module.

[0010] In this embodiment of the application, the input data of the input module includes: a first part of data, including historical sequence data of the capacity to be predicted and auxiliary prediction sequence data; a second part of data, which is static covariate data after preprocessing; and a third part of data, which is data obtained by the multilayer perceptron residual module for future known covariates.

[0011] In this embodiment of the application, the encoder is composed of multiple multilayer sensor residual modules stacked together.

[0012] In this embodiment, the multilayer perceptron residual module includes short connection branches and a main branch; the input data of the input module passes through a first fully connected layer on the short connection branch to obtain a first intermediate result vector; the input data of the input module passes through a second fully connected layer, a leaky rectifier linear unit activation function module, a third fully connected layer, and a random deactivation layer on the main branch to obtain a second intermediate result vector; the output vector of the input module is obtained by layer normalization based on the first intermediate result vector and the second intermediate result vector.

[0013] In this embodiment of the application, the capacity prediction model based on the classification-based basic data for a future preset time period is used to obtain the prediction result, which includes: using real-time indicator data, business data, related resource data and application performance data as input parameters, training the capacity prediction model, and obtaining the training result.

[0014] In this embodiment of the application, determining and pushing the predicted values ​​of each capacity indicator based on the prediction results includes: identifying the predicted values ​​of each capacity indicator based on the training results; comparing the predicted values ​​of each capacity indicator with the associated resource data under the corresponding capacity indicator to determine whether there is an over-limit capacity indicator; and when there is an over-limit capacity indicator, pushing information to the over-limit capacity indicator in a differentiated manner and triggering alarm information.

[0015] A second aspect of this application provides a capacity prediction system, comprising: a data acquisition unit for acquiring indicator data of the capacity to be predicted and preprocessing the indicator data to obtain preprocessed indicator data; wherein the indicator data includes: historical indicator data, real-time indicator data, business data, associated resource data, and application performance data; a processing unit for performing basic data classification based on the preprocessed indicator data; a prediction unit for performing capacity prediction for a preset time period in a capacity prediction model based on the basic data to obtain prediction results; and a push unit for determining and pushing the predicted values ​​of each capacity indicator based on the prediction results.

[0016] A third aspect of this application provides a processor configured to perform the capacity prediction method described above.

[0017] A fourth aspect of this application provides a machine-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the capacity prediction method described above.

[0018] The fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described capacity prediction method.

[0019] Through the above technical solution, when training the model, this invention uses historical data as input, and also incorporates related business data, upstream and downstream data, and network topology configuration data as auxiliary prediction data. The deep learning model fits the data distribution relationship to accurately predict future capacity trends. By comprehensively collecting and predicting relevant data, it avoids the problem of low training accuracy caused by existing solutions relying solely on single historical data for training.

[0020] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings:

[0022] Figure 1 A schematic flowchart of a capacity prediction method according to an embodiment of this application is shown.

[0023] Figure 2 This schematic diagram illustrates the structure of an input module according to an embodiment of the present application;

[0024] Figure 3 A schematic diagram of a multilayer sensor residual module according to an embodiment of this application is shown.

[0025] Figure 4 A schematic diagram illustrating the structure of a capacity prediction system according to an embodiment of this application is shown.

[0026] Figure 5 The diagram illustrates the internal structure of a computer device according to an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0028] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, and back), the directional indicators are only used to explain the relative positional relationship and movement of the components in a specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0029] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0030] To ensure stable system operation, it is necessary to ensure that the system capacity meets the demand. With the diversification of business types, the nonlinear characteristics of capacity prediction are becoming more and more obvious. If the accuracy of capacity prediction can be guaranteed, early warning, capacity cleanup and expansion based on the capacity prediction results can greatly ensure the stable operation performance of the system.

[0031] For capacity warnings, existing operations and maintenance teams typically rely on monitoring tools such as Zabbix, configuring user-focused metrics and alarm triggers. They monitor real-time capacity usage and passively expand capacity based on their own operational experience. Secondly, based on the historical operational experience of operations personnel, supervisors assess capacity expansion trends and manage capacity accordingly. Alternatively, machine learning methods can be used to predict capacity over a future period, and capacity management can be based on future capacity trends. The specific process for capacity prediction is as follows:

[0032] 1) Input data: Obtain single capacity data of the capacity indicator to be predicted within a preset time period before the current moment;

[0033] 2) Data preprocessing: Preprocess the time series data of the capacity indicator to be predicted within a preset time period, including outlier removal, judgment of whether the data length meets the model training requirements, and repair of missing values ​​of the indicator.

[0034] 3) Feature engineering: Perform data feature mining on the indicators to be predicted, including sequence periodicity, outlier periodicity, data volume change type, upper and lower bounds of data change, holiday effect, etc., and process the data according to the characteristics of time series data;

[0035] 4) Model training: The processed data of the volume index to be predicted is used as training data. The regression prediction model is trained, and the deviation between the model prediction results and the actual data is used to optimize the regression prediction model and retain the optimal model.

[0036] 5) Model prediction: Input the time series data of the capacity indicator to be predicted within a preset time period into the regression prediction model to make predictions and obtain the prediction results.

[0037] Current capacity forecasting methods mostly use regression prediction based on the capacity index to be predicted. These regression models are generally linear, making it difficult to represent highly complex data and susceptible to outliers. In the modeling process, current capacity forecasting methods typically use historical data of the capacity index to predict future capacity. This approach fails to consider interrelated factors such as business factors, upstream and downstream data, and network topology configuration data, resulting in a limited data set and an inability to effectively predict capacity changes caused by business variations, changes in related resources, and changes in upstream and downstream businesses. Furthermore, regression prediction methods do not incorporate time characteristics into their modeling. As a typical application of time series analysis and forecasting, time characteristics are a crucial factor influencing capacity trends.

[0038] To address the issue of low capacity prediction accuracy in existing solutions, this invention proposes a novel capacity prediction method. During model training, this method incorporates not only historical data as input but also related business data, upstream and downstream data, and network topology configuration data as auxiliary prediction data. A deep learning model is then used to fit the data distribution relationships, enabling accurate prediction of future capacity trends. By comprehensively collecting and predicting relevant data, this method avoids the low training accuracy problem inherent in existing solutions that rely solely on single historical data for training.

[0039] Figure 1 A schematic flowchart of a capacity prediction method according to an embodiment of this application is shown. Figure 1 As shown in one embodiment of this application, a capacity prediction method is provided, including the following steps:

[0040] Step S10: Collect indicator data of the capacity to be predicted, and preprocess the indicator data to obtain preprocessed indicator data.

[0041] Specifically, the indicator data includes historical indicator data, real-time indicator data, business data, related resource data, and application performance data; the related resource data includes one or more of the following: total CPU cores, allocated CPU resources, remaining allocable CPU resources, total memory, allocated memory resources, and remaining usable memory resources.

[0042] In this embodiment of the invention, capacity prediction is performed using multivariate time series data. In addition to using historical time series data of the capacity to be predicted, static covariates, related business data, related resource data, application performance data, etc. are introduced to help the model learn more information for capacity prediction and reduce the deviation between capacity prediction and actual results.

[0043] Specifically, the preprocessing of the indicator data includes: performing one or more of the following processes on the indicator data: outlier removal, deduplication, missing segment repair, and data removal; identifying time-series data, and extracting future covariates from the identified time-series data.

[0044] Step S20: Perform basic data classification based on the preprocessed indicator data.

[0045] Specifically, the solution of this invention requires both model training based on collected indicator data and subsequent capacity prediction based on real-time data. Therefore, it is necessary to differentiate the indicator data to ensure the effectiveness of subsequent data processing, reduce output processing time, and improve data processing efficiency. Based on this, the solution of this invention differentiates the data according to the specified indicator data, using each differentiated dataset as the foundation data.

[0046] Step S30: Based on the classification data, perform capacity prediction for a future preset time period in the capacity prediction model to obtain the prediction results.

[0047] Specifically, the method further includes: training a capacity prediction model, including: splitting historical data of the indicator into training data and validation data, wherein the training data is earlier than the validation data in time series; based on the training data, reading the corresponding auxiliary prediction time series, known future covariates, and static covariates for the same period, and forming a training set together with the training data; in a pre-constructed neural network, using the data of the same period as the validation data as the training target, training the model based on the training set to obtain an initial model; comparing the training results of the initial model with the validation data, correcting the model based on the deviation, and after each model correction, comparing the training results of the corrected model with the validation data until the difference between the two satisfies the error minimization condition, stopping the model correction, and using the corrected model obtained from the latest correction as the capacity prediction model.

[0048] Among these, known future covariates are date-related variables such as which day of the week, which month, or even whether it is a public holiday. These covariates change over time, but are known in the future to be predicted; that is, these covariates are actually known without prediction. Static covariates are covariates that remain constant relative to capacity, such as predicting the category to which capacity belongs (business, IaaS layer resources, PaaS layer resources, etc.).

[0049] In this invention, time features are introduced into the known future covariates. Variables related to dates, such as the hour of the day, day of the week, month, and even whether it is a public holiday, are input into the model as covariates. This provides effective features for capacity prediction modeling and improves the accuracy of capacity prediction. This invention uses multivariate time series for capacity prediction, not only using historical time series of the capacity to be predicted, but also introducing static covariates, related business data, related resource data, application performance data, etc., to help the model learn more information for capacity prediction and reduce the deviation between the predicted capacity and the actual results.

[0050] Preferably, the pre-built neural network includes an input module, an encoder, a decoder, and an output module.

[0051] Preferably, the input data of the input module includes: a first part of data, including historical sequence data of the capacity to be predicted and auxiliary prediction sequence data; a second part of data, which is static covariate data after preprocessing; and a third part of data, which is data obtained by the multilayer perceptron residual module for known future covariates.

[0052] In one possible implementation, such as Figure 2 The input data can be divided into three parts: First, the historical sequence of the data to be predicted and the auxiliary prediction sequence of length L are respectively passed through fully connected layers. Their results are stacked and passed through fully connected layers, and then sequentially fed into convolutional layers with stride C (C is a hyperparameter). The outputs of each convolutional layer are concatenated and fed into a fully connected layer to obtain the first part of the output. Second, the preprocessed static covariates serve as the second part of the data. Third, the known future covariates of length L+H (L is the length of the historical data, and H is the length of the predicted data) are passed through the residual module of a multilayer perceptron to obtain the third part of the data. These three parts of the data are concatenated to obtain the final input.

[0053] Preferably, the encoder is composed of multiple multilayer sensor residual modules stacked together.

[0054] Preferred, such as Figure 3The multilayer perceptron residual module includes two branches: a short connection branch and a main branch; the input data passes through a first fully connected layer on the short connection branch to obtain a first intermediate result vector; the input data passes through a second fully connected layer, a leaky rectifier linear unit activation function module, a third fully connected layer, and a random deactivation layer on the main branch to obtain a second intermediate result vector; the output vector of the input module is obtained by layer normalization based on the first intermediate result vector and the second intermediate result vector.

[0055] In one possible implementation, in the residual module of the multilayer perceptron, the input data is divided into two paths: one path passes through a fully connected layer, is activated by a Leaky-ReLU activation function, and then passes through another fully connected layer before being deactivated to obtain a result vector; the other path passes through a fully connected layer to obtain an intermediate result vector. The result vectors from the two paths are summed and then layer normalized to obtain the output vector.

[0056] In a fully connected layer, each node is connected to all nodes in the previous layer, combining the extracted features. Due to its fully connected nature, it typically has the most parameters. The Leaky Rectified Linear Unit (Leaky ReLU) is a specialization of the ReLU function; when x < 0, its function value is no longer equal to 0, but decreases slowly with a small slope. This activation function requires an α value; common values ​​are between 0.01 and 0.1. Random deactivation involves randomly ignoring or masking some neurons during training at a certain ratio (the ratio parameter can be set). These neurons are randomly "discarded," meaning their contribution to downstream neurons during forward propagation is temporarily lost, and their weights are not updated during backpropagation. Therefore, through propagation, dropout produces the same effect of shrinking weights as the L2 norm. Layer normalization normalizes all features for each sample. A convolutional layer in a convolutional neural network consists of several convolutional units, and the parameters of each convolutional unit are optimized through backpropagation. The purpose of convolution is to extract different features from the input. The first convolutional layer may only extract some low-level features such as edges, lines and corners. More layers of the network can iteratively extract more complex features from the low-level features.

[0057] In this embodiment of the invention, a nonlinear activation function is added after the fully connected layer during the construction of the capacity prediction model. This is done by superimposing a nonlinear activation function after a linear transformation to avoid the multi-layer network being equivalent to a single-layer linear function, thus enhancing the model's labeling ability, solving problems that linear models cannot address, and thereby achieving greater learning and fitting capabilities and improving the model's prediction accuracy. This invention introduces deep learning technology, using fully connected layers and convolutional layers as the foundation to construct a capacity prediction neural network, and introducing Leaky-ReLU to add a nonlinear factor to the model, increasing its nonlinear fitting ability. The proposed model achieves comparable or better prediction accuracy than recurrent neural networks and transformer-type models while being more efficient, making it more suitable for production applications.

[0058] Furthermore, in the decoder, the encoder output enters multiple stacked multilayer perceptron residual modules to obtain intermediate result one. The H portion of the result obtained from the multilayer perceptron residual module for future known covariates is truncated and used as intermediate result two. The two intermediate results are stacked and then sent to the multilayer perceptron residual module.

[0059] Specifically, real-time indicator data, business data, related resource data, and application performance data are used as input parameters to train the capacity prediction model and obtain training results. That is, based on historical data of the capacity to be predicted, auxiliary prediction time series (including business data, related resource data, application performance data, etc.), known future covariates, and static covariates, capacity prediction is performed through the aforementioned neural network model.

[0060] Step S40: Based on the prediction results, determine and push the predicted values ​​of each capacity indicator.

[0061] Specifically, based on the training results, the predicted values ​​of each capacity indicator are identified; the predicted values ​​of each capacity indicator are compared with the associated resource data under the corresponding capacity indicator to determine whether there is an over-limit capacity indicator; when there is an over-limit capacity indicator, information is pushed to the over-limit capacity indicator in a differentiated manner and an alarm is triggered.

[0062] In this embodiment of the invention, for example, if the predicted remaining CPU demand is greater than the remaining available CPU, an early warning needs to be issued so that users can clean up the CPU to release more available CPU and avoid system overload or disruption to normal user business operations.

[0063] In one embodiment, such as Figure 4As shown, a capacity prediction system is provided, wherein the system includes: a data acquisition unit for acquiring indicator data of the capacity to be predicted and preprocessing the indicator data to obtain preprocessed indicator data; a processing unit for performing basic data classification based on the preprocessed indicator data; a prediction unit for performing capacity prediction for a future preset time period in a capacity prediction model based on the basic data to obtain prediction results; and a push unit for determining and pushing the predicted values ​​of each capacity indicator based on the prediction results.

[0064] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and capacity prediction methods can be implemented by adjusting kernel parameters.

[0065] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0066] This application provides a storage medium storing a program that, when executed by a processor, implements the above-described capacity prediction method.

[0067] This application provides a processor for running a program, wherein the program executes the above-described capacity prediction method during runtime.

[0068] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The network interface A02 is used for communication with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a capacity prediction method.

[0069] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0070] In one embodiment, the capacity prediction system provided in this application can be implemented as a computer program, which can be implemented in the form of, for example, Figure 5 The system runs on the computer device shown. The memory of the computer device can store various program modules that make up the capacity prediction system, and the computer program composed of the various program modules causes the processor to execute the steps in the capacity prediction methods of the various embodiments of this application described in this specification.

[0071] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing the above-described capacity prediction method.

[0072] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0074] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0075] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0076] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0077] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0078] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0080] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A capacity prediction method, characterized in that, The method includes: Collect indicator data for the capacity to be predicted, and preprocess the indicator data to obtain preprocessed indicator data; wherein, the indicator data includes: historical indicator data, real-time indicator data, business data, related resource data, and application performance data; the related resource data includes one or more of the following: total number of CPU cores, allocated CPU, remaining allocable CPU, total memory, allocated memory, and remaining usable memory. Basic data classification is performed based on the preprocessed indicator data; Based on the classification-based data, capacity prediction for future preset time periods is performed in the capacity prediction model to obtain prediction results; among which, The training rules for the capacity prediction model are as follows: Historical data of the indicator is split into training data and validation data, with the training data being earlier in time than the validation data. Based on the training data, auxiliary prediction time series, known future covariates, and static covariates for the corresponding time periods are read and combined with the training data to form a training set. In a pre-constructed neural network, the data from the same time period as the validation data is used as the training target, and the model is trained based on the training set to obtain an initial model. The training results of the initial model are compared with the validation data, and the model is corrected based on the deviation. Each time a model correction is completed, the training results of the corrected model are compared with the validation data again until the difference between the two meets the error minimization condition. Model correction is then stopped, and the corrected model obtained from the latest correction is used as the capacity prediction model. The pre-built neural network includes: an input module, an encoder, a decoder, and an output module; The input data of the input module includes: a first part of data, including historical sequence data of the capacity to be predicted and auxiliary prediction sequence data; a second part of data, which is preprocessed static covariate data; and a third part of data, which is data obtained by the multilayer perceptron residual module for known future covariates. The historical sequence of the volume to be predicted and the auxiliary prediction sequence of length L are respectively passed through a fully connected layer. Their results are stacked and passed through a fully connected layer, and then sequentially fed into a convolutional layer with stride C. The outputs of each convolutional layer are concatenated and fed into a fully connected layer to obtain the first part of the output. The encoder is composed of multiple multilayer sensor residual modules stacked together; The multilayer perceptron residual module includes short connection branches and a backbone branch; the input data of the input module passes through a first fully connected layer on the short connection branch to obtain a first intermediate result vector; the input data of the input module passes through a second fully connected layer, a leaky rectifier linear unit activation function module, a third fully connected layer, and a random deactivation layer on the backbone branch to obtain a second intermediate result vector; the output vector of the input module is obtained by layer normalization based on the first intermediate result vector and the second intermediate result vector. In the decoder, the encoder output enters multiple stacked multilayer perceptron residual modules to obtain intermediate result one. The H part of the result of the multilayer perceptron residual module after the known future covariates are processed is taken as intermediate result two. The two intermediate results are stacked and sent to the multilayer perceptron residual module. Based on the forecast results, the predicted values ​​for each capacity indicator are determined and pushed out.

2. The method according to claim 1, characterized in that, The preprocessing of the indicator data includes: Perform one or more of the following processing on the indicator data: outlier removal, deduplication, missing segment repair, and data removal. Identify time-series data and perform future covariate extraction processing on the identified time-series data.

3. The method according to claim 1, characterized in that, The classification-based basic data is used to predict capacity for a future preset time period in the capacity prediction model, and the prediction results are obtained, including: The capacity prediction model is trained by taking real-time indicator data, business data, related resource data, and application performance data as input parameters to obtain training results.

4. The method according to claim 1, characterized in that, The process of determining and pushing predicted values ​​for each capacity indicator based on the prediction results includes: Based on the training results, the predicted values ​​of each capacity index are identified; Compare the predicted values ​​of each capacity indicator with the associated resource data under the corresponding capacity indicator to determine whether there are any over-limit capacity indicators. When there are over-limit capacity indicators, differentiated information is pushed to the over-limit capacity indicators and alarm information is triggered.

5. A capacity prediction system, characterized in that, The system is used to execute the capacity prediction method according to any one of claims 1-4, and the system comprises: The data acquisition unit is used to collect indicator data of the capacity to be predicted and to preprocess the indicator data to obtain preprocessed indicator data; wherein, the indicator data includes: historical indicator data, real-time indicator data, business data, related resource data and application performance data; The processing unit is used to perform basic data classification based on the preprocessed indicator data; The prediction unit is used to perform capacity prediction for a preset time period in the capacity prediction model based on the basic data, and obtain the prediction result. The push unit is used to determine and push the predicted values ​​of each capacity indicator based on the prediction results.

6. A processor, characterized in that, It is configured to perform the capacity prediction method according to any one of claims 1 to 4.

7. A machine-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, this instruction causes the processor to be configured to perform the capacity prediction method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the capacity prediction method according to any one of claims 1 to 4.

Images