Methods, computer devices, and computer programs for accurate prediction and utilization of future system usage.

Abstract

To provide a method, a computer device, and a computer program for accurately predicting and utilizing future system usage.SOLUTION: A future system usage forecasting method includes the steps of: constructing a forecasting model for an input indicator using time series data of the input indicator stored for a first period of time; acquiring a correlation between the input indicator and a system performance indicator using time series data of the input indicator and the system performance indicator stored for a second period of time; forecasting the input indicator associated with at least one future point in time through the forecasting model for the at least one future point in time; and calculating the system performance indicator at the future point in time using the forecasted input indicator and the correlation.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The following description relates to a technique for predicting system usage.

Background Art

[0002] A cluster is a system that connects independent computer systems such as personal computers by network equipment for special purposes and is widely used as a solution for providing high performance computing and high availability computing.

[0003] This has the advantage that upgrades and expansions are possible by simply replacing or adding some nodes and networks without changing the entire system, providing flexible scalability. Due to such advantages, clusters are constructed and used in enterprises, research institutes, university laboratories, etc. that require complex scientific computing problems according to the required computing requirements.

[0004] For such clusters, monitoring technologies are required to continuously observe and maintain the normal operation and quality of the clusters.

[0005] For example, Patent Document 1 (publication date: June 16, 2009) discloses a large-scale cluster monitoring technology that can actively take measures against configuration information when a node failure occurs in a cluster system environment and can continuously improve availability.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

[0007] As input indicators that correlate with system performance metrics, we construct a time series forecasting model using input indicators that have few sudden change elements.

[0008] Instead of directly predicting system performance metrics, a predictive model is used to forecast input metrics at future points in time, and then the correlation-based system performance metrics are calculated.

[0009] Future system performance metrics will be used in conjunction with a system that automates resource management, such as the deployment and removal of equipment (machines). [Means for solving the problem]

[0010] A method for predicting future system usage of a computer device including at least one processor is provided, comprising the steps of: constructing a predictive model for input indicators using time-series data of input indicators stored over a first fixed period using at least one processor; determining the correlation between input indicators and system performance indicators using time-series data of input indicators and system performance indicators stored over a second fixed period using at least one processor; predicting input indicators for at least one future point in time using the predictive model with at least one processor; and calculating system performance indicators for the future point in time using the predicted input indicators and their correlations with at least one processor.

[0011] One aspect of this approach is that the input indicators are selected from among the indicators to which system resources respond, and whose change over a third fixed period is below a threshold.

[0012] From another perspective, the system is characterized by using user traffic, which corresponds to log data taken from an external source, as an input metric, and using CPU usage or disk usage as a system performance metric.

[0013] Another aspect of this approach involves using a linear regression model to determine the correlation between input metrics and system performance metrics.

[0014] From another perspective, the step of determining the correlation may include a step of determining the linear regression coefficient and linear regression intercept by linear regression on the time series data for a second fixed period.

[0015] Another aspect of this approach is that the linear regression intercept value is determined to be close to 0.

[0016] From another perspective, the stage of building the predictive model may include a stage of updating the predictive model using data stored after the previous prediction point for the input indicator.

[0017] From another perspective, the step of determining the correlation may include a step of updating the correlation between the input indicator and the system performance indicator at regular intervals.

[0018] From another perspective, the step of determining correlation is characterized by, depending on the type of system performance indicator, using the value obtained by integrating the time-series data of the input indicator over time to determine the correlation between the input indicator and the system performance indicator.

[0019] Another aspect suggests that the method for predicting future system usage may further include a step in which at least one processor uses a calculated system performance index to recommend a component configuration at a future point in time.

[0020] According to another aspect, the step of calculating the future system performance index calculates the future system performance index of each component constituting the cluster, and the step of recommending the future component configuration calculates and recommends the number of units of the corresponding component to be installed based on the future system performance index of each component.

[0021] According to another aspect, the future system usage prediction method may further include, by at least one processor, when a reception-scheduled input index is given for constructing a new cluster, calculating the system performance index of each component constituting the new cluster using the correlation with the reception-scheduled input index, and, by at least one processor, calculating and recommending the required number of units of each component based on the system performance index of each component.

[0022] According to yet another aspect, the future system usage prediction method may further include, by at least one processor, providing a time point at which the calculated system performance index exceeds the threshold range in the future.

[0023] Provide a computer program for causing a computer device to execute the future system usage prediction method.

[0024] Provide a computer device including at least one processor realized to execute instructions readable by the computer device, and by the at least one processor, constructing a prediction model of the input index using the time-series data of the input index stored during a first fixed period, obtaining the correlation between the input index and the system performance index using the time-series data of the input index and the system performance index stored during a second fixed period, predicting the input index at a future time point using the prediction model for at least one future time point, and calculating the system performance index at the future time point using the correlation with the predicted input index.

Advantages of the Invention

[0025] According to an embodiment of the invention, as an input index having a correlation with a system performance index, a prediction model for predicting a change in the input index is constructed by causing a time series prediction model to learn an input index with few sudden change elements, thereby reducing outliers of learning data and improving the accuracy of prediction.

[0026] According to an embodiment of the invention, rather than directly predicting a system performance index, a change in an input index is predicted using a prediction model, and a system performance index based on a correlation is calculated using a predicted value of the input index, whereby future system performance can be easily predicted only by the input index.

[0027] According to an embodiment of the invention, by utilizing a future system performance index in conjunction with a system that automates resource operations such as equipment installation and exclusion, the system can be stably maintained and managed, and the operation resources required for calculating the number of equipment when configuring a new cluster can be reduced.

Brief Description of the Drawings

[0028] [Figure 1] It is a diagram showing an example of a network environment in an embodiment of the present invention. [Figure 2] It is a block diagram showing an example of a computer device in an embodiment of the present invention. [Figure 3] It is a diagram showing an example of a cluster environment system in an embodiment of the present invention. [Figure 4] It is a flowchart showing an example of a method that can be executed by a computer device in an embodiment of the present invention. [[ID=B]] [Figure 5] It is a diagram showing an example of an input index and a system performance index in an embodiment of the present invention. [Figure 6] It is a diagram showing an example of an input index and a system performance index in an embodiment of the present invention. [Figure 7] It is a flowchart showing a prediction process of CPU usage in an embodiment of the present invention. [Figure 8] This figure shows an example of CPU usage prediction results in one embodiment of the present invention. [Figure 9] This figure shows an example of a configuration file for predicting resource usage in one embodiment of the present invention. [Figure 10] This figure shows an example of recommending a component configuration based on resource usage prediction in one embodiment of the present invention. [Figure 11] This figure shows an example of recommending a component configuration based on resource usage prediction in one embodiment of the present invention. [Figure 12] This figure shows an example of recommending a component configuration based on resource usage prediction in one embodiment of the present invention. [Modes for carrying out the invention]

[0029] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] Embodiments of the present invention relate to a technique for predicting system usage.

[0031] Embodiments specifically disclosed herein can provide accurate predictions for future system usage by calculating future system performance indicators based on predicted changes in input indicators. This is achieved by training a time-series prediction model with input indicators that have few sudden change elements, which are correlated with system performance indicators, to predict changes in the input indicators.

[0032] A future system usage forecasting device according to an embodiment of the present invention may be implemented by at least one computer device, and a future system usage forecasting method according to an embodiment of the present invention may be executed by at least one computer device included in the future system usage forecasting device. In this case, a computer program according to one embodiment of the present invention may be installed and executed in the computer device, and the computer device may execute the future system usage forecasting method according to an embodiment of the present invention in accordance with the control of the executed computer program. The above-described computer program may be recorded on a computer-readable recording medium in conjunction with the computer device to cause the computer to execute the future system usage forecasting method.

[0033] Figure 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in Figure 1 shows an example that includes a plurality of electronic devices 110, 120, 130, 140, a plurality of servers 150, 160, and a network 170. Figure 1 is merely an example for the purpose of explaining the invention, and the number of electronic devices and servers should not be limited to that shown in Figure 1. Furthermore, the network environment in Figure 1 is merely an example for the purpose of explaining an environment applicable to this embodiment, and the environment applicable to this embodiment should not be limited to the network environment in Figure 1.

[0034] The multiple electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented by computer devices. Examples of the multiple electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, PCs (personal computers), notebook PCs, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), and tablets. As an example, Figure 1 shows a smartphone as an example of electronic device 110, but in embodiments of the present invention, electronic device 110 may mean one of a variety of physical computer systems that can communicate with other electronic devices 120, 130, 140 and / or servers 150, 160 via the network 170 using substantially wireless or wired communication methods.

[0035] The communication method is not limited, and may include not only communication methods that utilize communication networks that can be included in network 170 (for example, mobile communication networks, wired internet, wireless internet, broadcasting networks), but also short-range wireless communication between devices. For example, network 170 may include one or more arbitrary networks such as PAN (personal area network), LAN (local area network), CAN (campus area network), MAN (metropolitan area network), WAN (wide area network), BBN (broadband network), and the Internet. Furthermore, network 170 may include, but is not limited to, one or more network topologies, including bus networks, star networks, ring networks, mesh networks, star-bus networks, tree or hierarchical networks.

[0036] Servers 150 and 160 may each be implemented by one or more computer devices that communicate with multiple electronic devices 110, 120, 130, and 140 via the network 170 to provide commands, code, files, content, services, etc. For example, server 150 may be a system that provides services (for example, a log integration management service) to multiple electronic devices 110, 120, 130, and 140 connected via the network 170.

[0037] Figure 2 is a block diagram showing an example of a computer device in one embodiment of the present invention. Each of the aforementioned electronic devices 110, 120, 130, and 140, as well as each of the servers 150 and 160, may be implemented by the computer device 200 shown in Figure 2.

[0038] Such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240, as shown in Figure 2. The memory 210 is a computer-readable recording medium and may include RAM (random access memory), ROM (read-only memory), and persistent mass storage devices such as disk drives. Here, persistent mass storage devices such as ROM and disk drives may be included in the computer device 200 as separate persistent storage devices distinct from the memory 210. The memory 210 may also store an operating system and at least one program code. Such software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such a separate computer-readable recording medium may include computer-readable recording media such as floppy disks, disks, tapes, DVD / CD-ROM drives, and memory cards. In other embodiments, the software components may be loaded into the memory 210 through a communication interface 230, which is not a computer-readable recording medium. For example, software components may be loaded into the memory 210 of the computer device 200 based on computer programs installed by files received via the network 170.

[0039] The processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor 220 by memory 210 or a communication interface 230. For example, the processor 220 may be configured to execute instructions received according to program code stored in a recording device such as memory 210.

[0040] The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (for example, the recording device described above) via the network 170. For example, requests, instructions, data, files, etc., generated by the processor 220 of the computer device 200 according to program code recorded in a recording device such as memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data, files, etc., from other devices may be received by the computer device 200 via the network 170 through the communication interface 230 of the computer device 200. Signals, instructions, data, etc., received via the communication interface 230 may be transmitted to the processor 220 or memory 210, and files, etc., may be recorded on a recording medium (the persistent recording device described above) that the computer device 200 may further include.

[0041] The input / output interface 240 may be a means for interface with the input / output device 250. For example, the input device may include a microphone, keyboard, or mouse, and the output device may include a display or speaker. In another example, the input / output interface 240 may be a means for interface with a device that integrates input and output functions into one, such as a touchscreen. The input / output device 250 may consist of a single device together with the computer device 200.

[0042] In other embodiments, the computer device 200 may include fewer or more components than those shown in Figure 2. However, it is not necessary to explicitly show most of the conventional components in the figure. For example, the computer device 200 may be implemented to include at least some of the input / output devices 250 described above, and may further include other components such as transceivers and databases.

[0043] The following describes specific implementations of technologies for accurate prediction and utilization of future system usage.

[0044] This embodiment targets a cluster system environment and includes a technology for predicting system usage and a technology for utilizing the prediction results of system usage in system operation.

[0045] In this specification, a cluster refers to a system that receives and processes large amounts of data, and Figure 3 shows an example of a cluster system.

[0046] Referring to Figure 3, the integrated log management system 300 is a system that receives logs 301 from various services and supports their storage in the real-time indexing and search system 310 and the data storage platform 320. In other words, the integrated log management system 300 is a platform that collects logs 301 generated from various services in one place and makes them available for use in problem tracking and data analysis.

[0047] The Integrated Log Management System 300 is comprised of an open-source framework that processes large amounts of data in a distributed environment, storing it by dividing and replicating it across multiple disks on multiple clustered computers.

[0048] A cluster of such an integrated log management system 300 consists of multiple components, each component consisting of at least one piece of equipment (for example, a server).

[0049] In the following embodiments, an integrated log management system 300, which consists of multiple components, is described as a typical example of a cluster. However, the system is not limited to this, and can be expanded to any system environment that consumes resources in response to input data provided from an external source.

[0050] The computer device 200 according to this embodiment may be configured with a computer-implemented future system usage prediction device. The processor 220 of the computer device 200 may be implemented by components for performing the following future system usage prediction method. Depending on the embodiment, the components of the processor 220 may be selectively included in or excluded from the processor 220. Also, depending on the embodiment, the components of the processor 220 may be separated or merged in order to express the functions of the processor 220.

[0051] Such a processor 220 and its components may control the computer device 200 to perform steps included in the following future system usage prediction method. For example, the processor 220 and its components may be implemented to execute instructions from the operating system code contained in the memory 210 and the code of at least one program.

[0052] Here, the components of the processor 220 may be representations of different functions that are executed by the processor 220 in accordance with instructions provided by the program code recorded in the computer device 200.

[0053] The processor 220 may read necessary instructions from the memory 210, which is loaded with instructions related to the control of the computer device 200. In this case, the instructions read may include instructions for controlling the processor 220 to perform the steps described below.

[0054] The steps included in the future system usage forecasting method described below may be performed in a different order than shown in the diagram, and some steps may be omitted or additional processes may be included.

[0055] Figure 4 is a flowchart showing an example of a method that a computer device can perform in one embodiment of the present invention.

[0056] Referring to Figure 4, in step 410, the processor 220 may construct a predictive model for the input metric using time-series data of the pre-selected input metric. The processor 220 may generate a time-series predictive model for the input metric by using time-series data of the input metric stored over a relatively long period as training data and training a time-series prediction library (e.g., Prophet) with this data.

[0057] In this embodiment, the input indicator may be an external indicator that does not undergo sudden changes, as an indicator to which system resources respond. Input and output indicators for the system may be analyzed over a long period of time, and indicators whose short-term changes are below a threshold may be selected to be used as training data for the prediction model. System performance indicators such as CPU and disk usage have the characteristic of undergoing large changes in a short period of time due to parameter tuning, rollbacks, and failure conditions, and using such indicators as training data can lead to large prediction errors. On the other hand, user traffic, which is log data taken in from an external source and is one of the input indicators, does not change much in the short period of time, so if it is used as training data for the prediction model, outliers in the training data can be reduced and the accuracy of the prediction can be improved. As an example, the processor 220 may construct a prediction model for predicting traffic changes by retrieving traffic time series data stored over a long period of time and training a time series prediction library with the retrieved traffic data.

[0058] At stage 420, processor 220 may determine the correlation between input metrics and system performance metrics. System performance metrics are output metrics that have characteristics that respond to input metrics, and may refer to output metrics that indicate system performance, such as CPU usage (number of cores used per second), disk usage (disk capacity in use), and memory usage (memory capacity in use). System performance metrics such as CPU usage, disk usage, and memory usage are related to user traffic, which is one of the input metrics. For example, in the case of a search service, the correlation between search traffic and the CPU / disk usage of the search server may be utilized, and in the case of a blog service, the correlation between usage traffic such as blog search, posting, and neighbor notifications and the CPU / disk usage of each blog server may be utilized. In the case of a shopping service, the correlation between usage traffic such as shopping search, purchase, and shipping inquiry and the CPU / disk usage of each shopping server may be utilized. The processor 220 retrieves short-term (e.g., 3 days) time-series data for both the input indicator and the system performance indicator, and uses a linear regression model to derive the correlation between the input indicator and the system performance indicator. In this embodiment, the correlation between the input indicator and the system performance indicator is described as being derived by linear regression, but this is not the only method that can define the correlation between the input indicator and the system performance indicator from time-series data, and any such technique is applicable.

[0059] In step 430, the processor 220 may use a prediction model to predict the input indicators for at least one future point in time (e.g., 7 days later, 30 days later, etc.), and then use the predicted input indicators and the correlation obtained in step 420 to calculate the system performance indicators for the future point in time. In this embodiment, instead of directly predicting system performance indicators that have many sudden change elements, the prediction model constructed in step 410 is used to predict future changes in input indicators that have few sudden change elements. At this time, the processor 220 may apply the correlation between the input indicators and the system performance indicators to the input indicators for the future point in time predicted using the prediction model to calculate the system performance indicators for the future point in time. In other words, the processor 220 predicts changes in the input indicators for each future time point in time in advance by time series prediction, and then uses the predicted input indicators and the previously obtained correlation to predict future changes in the system performance indicators.

[0060] At stage 440, processor 220 may recommend a component configuration at a given point in time based on system performance indicators at that point in time. For example, processor 220 may predict the future system performance indicators of each component constituting the cluster, assuming that the trend for the input indicators is maintained. If the predicted future value of the system performance indicator exceeds a threshold range, it may recommend adding equipment to the relevant component. If the predicted future value of the system performance indicator falls below the threshold range, it may recommend partially removing equipment from the relevant component. In order to configure an efficient cluster, the number of components to add or remove is calculated and recommended based on the predicted future change in the system performance indicators. As another example, when a new cluster is added and new linkages are made, if the predicted values ​​of the input indicators generated by the new linkage are given, processor 220 may recommend the number of components to each component in that cluster. When building a new cluster, the number of components required for each component is calculated and recommended.

[0061] Figures 5 and 6 show examples of input metrics and system performance metrics in one embodiment of the present invention.

[0062] Most services have two main needs: to provide stable service by deploying more equipment when resource usage is too high, and to reduce operating costs by removing some existing equipment when resource usage is too low, depending on the resource usage of each piece of equipment.

[0063] To achieve stable system operation and reduce operational resources, accurate predictions of future system usage are necessary. This requires building highly accurate models and considering the correlation between input metrics and system performance metrics.

[0064] Referring to Figure 5, in this embodiment, user traffic 501, which indicates the amount of log data collected by service users (e.g., BPS (bytes / s), QPS (query / s), etc.), may be used as an input indicator, and the CPU usage 502 of a component that responds to such user traffic 501 may be used as a system performance indicator. Time-series data of user traffic 501 and CPU usage 502 stored over a long period of time shows that the relationship between user traffic 501 and CPU usage 502 is extremely high.

[0065] Referring to Figure 6, not only CPU usage 502 but also disk usage 602 are system performance indicators that respond to user traffic 501. Based on time-series data of user traffic 501 and disk usage 602 stored over a long period, it can be seen that the value 601 obtained by integrating the BPS corresponding to user traffic 501 over time is similar to the disk usage 602. This indicates that the relationship between user traffic 501 and disk usage 602 is extremely high.

[0066] Figure 7 is a flowchart showing the CPU usage prediction process in one embodiment of the present invention.

[0067] Referring to Figure 7, the processor 220 may retrieve time-series data of user traffic that has been stored for a relatively long period of time as training data for the prediction model (step 701).

[0068] The processor 220 may generate a time series forecast model for user traffic by training a time series forecasting library (e.g., Prophet) with the traffic data retrieved in step 701 (step 702).

[0069] The processor 220 may retrieve time-series data (X) of user traffic that has been stored for a relatively shorter period than the model training data (step 703).

[0070] Processor 220 may retrieve time-series data (Y) of CPU usage that has been stored for a relatively shorter period than the model training data (step 704).

[0071] Processor 220 may use the time-series data of user traffic and CPU usage retrieved in steps 703 and 704 to determine the correlation between user traffic and CPU usage (Y=aX+b) using a linear regression model (step 705). Here, a represents the linear regression coefficient and b represents the linear regression intercept, and a and b may be determined by linear regression on short-term time-series data. For example, time-series data stored for 3 days may be used to determine a and b, and this period can be adjusted as needed.

[0072] In the correlation between user traffic and CPU usage, b can be determined to be a value close to 0. Setting b to 0 might also make sense, but since CPU usage in an idle state is not always 0, b should not be set to 0.

[0073] The processor 220 may, after predicting user traffic (X') at a future time (t) using the time series prediction model constructed in step 702, calculate the CPU usage (Y') at the future time (t) using the future predicted value (X') for user traffic and the correlation obtained in step 705 (step 706).

[0074] When adding or removing equipment to a component, or performing server tuning, actual CPU usage can change significantly in a short period. If CPU usage is used as training data, such changes may increase prediction errors, but user traffic rarely changes in a short period. In this embodiment, a time-series prediction model is constructed using user traffic, which has few sudden changing elements. Furthermore, by determining the correlation between user traffic and the CPU usage of equipment, it is possible to derive a relationship such as how much CPU is used for a given traffic pattern. By using a prediction model on user traffic, it is possible to predict how much (Δ) user traffic (X) will change in the future, and from the correlation between user traffic and CPU usage, it is possible to predict how much (aΔ) of CPU will be used if user traffic (X) changes by Δ. Figure 8 shows an example of predicting CPU usage in one embodiment of the present invention. In Figure 8, 801 shows the CPU usage (Y) correlated with user traffic (aX+b), 802 shows the value (Y') obtained by multiplying the value (X') predicted by the time series prediction model that learned user traffic (X) by a and adding b, 803 shows the prediction error (upper / lower) of the time series prediction model, and 804 shows the actually measured CPU usage.

[0075] The learning process of the prediction model (stage 702) and the correlation analysis process (stage 705) may be executed at regular intervals. In this case, if user traffic (X) is added after the previous prediction point in time, the model may be periodically trained and updated to reflect the added data in the prediction model. Also, even if the user traffic (X) is the same, if the CPU usage (Y) increases or decreases due to parameter tuning or rollback, it is necessary to periodically recalculate the correlation between user traffic (X) and CPU usage (Y) in order to reflect such changes in the prediction process (stage 706). For example, in the relationship Y = aX + b between user traffic (X) and CPU usage (Y), even if the traffic is the same, the correlation may change from a=0.1, b=1 before tuning to a=0.05, b=1 after tuning due to parameter tuning. By periodically updating the correlation between user traffic (X) and CPU usage (Y), it is possible to respond to changes in the relationship and predict CPU usage at future points in time more accurately.

[0076] Disk usage, another system performance indicator, can also be predicted to change in the future using the same method as the CPU usage prediction process explained in Figure 7. However, the correlation between user traffic and disk usage is determined by retrieving short-term time-series data of user traffic and utilizing the value obtained by integrating it over time.

[0077] The correlation between user traffic (X) and disk usage (Y) may be defined as follows: Disk usage (Y) = a × [integral value of user traffic (X) stored during retention time × number of copies] + b. In the correlation between user traffic and disk usage, a may be determined to be a value close to 1. Depending on the embodiment, it may be meaningful to set a to 1 and b to 0.

[0078] By determining the correlation between user traffic and disk usage, we can derive a relationship that indicates how much disk space will be used for a given traffic level. By using a predictive model on user traffic, we can predict how much (Δ) user traffic (X) will change in the future. Furthermore, based on the correlation between user traffic and disk usage, we can predict how much (aΔ) of disk space will be used if user traffic (X) changes by Δ.

[0079] Figure 9 shows an example of a configuration file for predicting resource usage in one embodiment of the present invention. As an example of a cluster, Figure 9 shows a configuration file for predicting resource availability for a system called Logiss, a log integration management platform published by NAVER. The future system usage prediction technology of the present invention described above can be abstracted as shown in the configuration file in Figure 9 so that it can be used in other services and additional models can be added.

[0080] For example, referring to Figure 9, the Logiss system capacity prediction may include a query file 901 for calculating user traffic, a query file 902 for calculating CPU usage, a model file 903 for analyzing the correlation between user traffic and CPU usage, a file 904 for setting query execution targets for resource capacity prediction, an integer 905 indicating the interval for performing linear regression of the two query files 901 and 902 specified by model file 903, an integer 906 indicating the period for comparing actual CPU usage with the value obtained by linear regression (Y=aX+b), and an integer 907 indicating the minimum / maximum range of the training data used for traffic prediction. The configuration file for disk usage prediction may also be similar to or identical to the configuration file in Figure 9, and high reusability can be provided by the abstracted contents of the configuration file.

[0081] In short, it is possible to obtain a correlation between user traffic (X) and CPU / disk usage (Y) from time-series data stored over a short period, and thereby calculate changes in CPU / disk usage due to future traffic changes. In this case, instead of directly predicting CPU and disk usage, outliers in the training data can be reduced by first predicting only the changes in traffic and then using the correlation (coefficient and intercept) to calculate the changes in CPU / disk usage due to the predicted traffic changes.

[0082] By selecting an input metric with few sudden fluctuations and determining its correlation with system performance metrics, this can be applied to various services. In this case, to apply resource usage prediction to various services, settings for queries, models, TSDB targets, etc., may be abstracted.

[0083] Figures 10 to 12 illustrate an example of recommending a component configuration based on resource usage prediction in one embodiment of the present invention.

[0084] The processor 220 may, for each component constituting the cluster, calculate future system performance metrics using a predictive model for the input metrics and the correlation between the input metrics, and then recommend a component configuration based on the future system performance metrics.

[0085] To facilitate the prediction of resource usage for each component, each component may be quantified as a unit system. For example, 4 CPUs, 500G (disk), and 32G (memory) capacity may be defined as one unit system, and the total resources required may be expressed as the number of these quantified unit systems.

[0086] Component configuration recommendations may include recommendations on how many components to add or remove if future CPU / disk usage is predicted to exceed or fall below a predetermined threshold range, or recommendations on the number of components needed to withstand a certain level of system resources (e.g., maximum 800 MiB / s + 10 TiB of data ingestion per day) based on the correlation between user traffic and CPU / disk usage.

[0087] The processor 220 may link each component constituting the cluster to a system that automates operations such as deploying and removing equipment, and may provide recommendation information regarding the component configuration through an administrator screen.

[0088] Figure 10 shows the Availability Status Administrator screen 1000, which recommends component configurations according to the availability status of the integrated log management system 300. Referring to Figure 10, when the spec information 1010 of a specific cluster and the target resource usage 1020 are entered into the Availability Status Administrator screen 1000, the processor 220 recommends the number of units 1040 required for each component based on the predicted value 1030 for resource usage at a future point in time 1001 and the predicted value 1030 for target resource usage 1020. Therefore, by inputting equipment specifications such as the number of CPU cores and disk capacity, it is possible to calculate and recommend how many units of equipment should be deployed to match the target range desired by the operator in preparation for future traffic changes. For example, when the automated equipment operation system targets a system utilization rate of 60% or less and 5% or more within 7 or 30 days as predicted by this system, it can receive the recommended number of units to deploy or exclude via API and appropriately adjust the component configuration for operation.

[0089] Figure 11 shows the capacity prediction administrator screen 1100, which recommends component configurations based on capacity predictions due to traffic additions to the integrated log management system 300. Referring to Figure 11, when the processor 220 receives the specifications 1110 of a specific cluster, the target resource usage 1120, and the amount of traffic to be added 1150 into the capacity prediction administrator screen 1100, it recommends the required number of each component based on the predicted value 1130 for resource usage at a future point in time 1101 due to the traffic addition, and the predicted value 1130 for the target resource usage 1120. Even in environments where new traffic is added, it can calculate and recommend how many units of equipment should be deployed to match the target range desired by the operator in preparation for future traffic changes.

[0090] In the embodiments described above, we explained how to predict the system performance indicators of each component for future time points 1001 and 1101 and recommend the component configuration for those time points. However, we are not limited to this, and depending on the embodiment, it is also possible to predict the system performance indicators for future time points at regular intervals and provide a UI that allows administrators to check future time points where the predicted value of the system performance indicator exceeds a threshold range (e.g., target resource usage) (i.e., time points where it is expected to exceed or fall below the threshold range). In other words, in the process of utilizing the system capacity prediction results, it is possible to guide administrators to future time points where the system capacity exceeds a predetermined threshold range, and to help them proactively adjust the number of components equipped during periods of low traffic.

[0091] Figure 12 shows the new cluster construction administrator screen 1200, which recommends a component configuration for building a new cluster of the integrated log management system 300. Referring to Figure 12, when the spec information 1210 of the cluster to be added, the target resource usage 1220, and the scale of the expected incoming traffic (e.g., peak traffic value, amount of traffic taken in per day) 1230 are entered into the new cluster construction administrator screen 1200, the processor 220 may recommend the required number of each component 1240 that can handle the expected incoming traffic scale 1230 within the target resource usage 1220. Based on the correlation between CPU usage (number of cores used per second) = aX (peak traffic value) + b, a and b are calculated for each component. When the peak traffic value received by the cluster is entered, the maximum CPU usage and the number of cores used per second are calculated, so the processor can calculate and recommend how many units of equipment are needed to keep the maximum CPU usage below the target range. Similarly, the correlation between disk usage and the total disk usage is calculated from the amount of traffic the cluster takes in per day, using the formula: disk usage = a x (integral value of traffic x number of copies) + b. This makes it possible to calculate how many devices need to be deployed to keep disk usage below the target range.

[0092] Thus, according to embodiments of the present invention, by training a time-series prediction model with input indicators that have a correlation with system performance indicators and have few sudden change elements, it is possible to construct a prediction model for predicting changes in input indicators, thereby reducing outliers in the training data and improving the accuracy of predictions. Furthermore, according to embodiments of the present invention, instead of directly predicting system performance indicators, changes in input indicators are predicted using a prediction model, and system performance indicators are calculated based on the correlation using the predicted values ​​of the input indicators, making it possible to easily predict future system performance using only input indicators. Moreover, according to embodiments of the present invention, by linking future system performance indicators to a system that automates resource management such as the deployment and removal of equipment, it is possible to stably maintain and manage the system, and to reduce the operational resources required to calculate the number of equipment when configuring a new cluster.

[0093] The apparatus described above may be implemented by hardware components, software components, and / or combinations of hardware and software components. For example, the apparatus and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as processors, controllers, ALUs (arithmetic logic units), digital signal processors, microcomputers, FPGAs (field programmable gate arrays), PLUs (programmable logic units), microprocessors, or various devices capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications running on the OS. The processing unit may also respond to software execution, access data, record, manipulate, process, and generate data. For convenience of understanding, it may be described as if a single processing unit is used, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0094] Software may include computer programs, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or which may instruct the processing unit independently or collectively. Software and / or data may be embodied in any kind of machine, component, physical device, computer recording medium, or device for interpretation based on the processing unit or for providing instructions or data to the processing unit. Software may be distributed across a network of computer systems, and may be recorded and executed in a distributed manner. Software and data may be recorded on one or more computer-readable recording media.

[0095] The methods according to the embodiment may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. In this case, the medium may continuously record computer-executable programs or may temporarily record them for execution or download. Furthermore, the medium may be various recording or storage means in the form of a combination of one or more hardware components, and may be a medium directly connected to a computer system or distributed on a network. Examples of mediums include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and devices configured to record program instructions such as ROM, RAM, and flash memory. Other examples of mediums include recording media and storage media managed by app stores that distribute applications, and sites and servers that supply and distribute various other software.

[0096] As described above, embodiments have been explained based on limited embodiments and drawings, but those skilled in the art will be able to make various modifications and variations from the above description. For example, the described technique may be performed in a different order than described, and / or the components of the described system, structure, apparatus, circuit, etc. may be combined or assembled in a different manner than described, or opposed or replaced by other components or equivalents, and still achieve suitable results.

[0097] Therefore, even if the embodiment is different, if it is equivalent to the claims, it falls within the scope of the attached claims. [Explanation of symbols]

[0098] 300: Integrated Log Management System 310: Real-time indexing and search system 320: Data Storage Platform

Claims

1. A method for predicting the future system usage of a computer device including at least one processor, The step of constructing a predictive model for the input indicator using time-series data of the input indicator stored over a first fixed period using at least one of the aforementioned processors, A step of determining the correlation between the input indicator and the system performance indicator using time-series data of the input indicator and the system performance indicator stored in a second fixed period by at least one of the processors, The steps include: using the at least one processor to predict the input indicator for at least one future time point using the prediction model; and The step of using the at least one processor to calculate the system performance indicator at a future point in time using the predicted input indicator and the correlation. Includes, The aforementioned input indicators are selected from among the indicators to which system resources respond, in which the change over a third fixed period is less than a threshold. The aforementioned input metric is user traffic corresponding to log data ingested from an external source. A method for predicting future system usage, wherein the system performance indicator is either CPU usage or disk usage.

2. The step of determining the correlation is as follows: The correlation between the input indicator and the system performance indicator is determined using a linear regression model. A method for predicting future system usage according to claim 1, characterized by the above.

3. The step of determining the correlation is as follows: The second step involves determining the linear regression coefficient and linear regression intercept values ​​by performing linear regression on the time series data over a fixed period. A method for predicting future system usage according to claim 1, including the following:

4. A method for predicting future system usage of a computer device including at least one processor, The step of constructing a predictive model for the input indicator using time-series data of the input indicator stored over a first fixed period using at least one of the aforementioned processors, A step of determining the correlation between the input indicator and the system performance indicator using time-series data of the input indicator and the system performance indicator stored in a second fixed period by at least one of the processors, The steps include: using the at least one processor to predict the input indicator for at least one future time point using the prediction model; and The step of using the at least one processor to calculate the system performance indicator at a future point in time using the predicted input indicator and the correlation. Includes, The step of determining the correlation is as follows: The second step involves determining the linear regression coefficient and linear regression intercept values ​​by performing linear regression on the time series data over a fixed period. Includes, The linear regression intercept value is determined to be close to zero, in this method for predicting future system usage.

5. The step of constructing the aforementioned predictive model is, The step of updating the prediction model using data stored after the previous prediction point for the input indicator. A method for predicting future system usage according to claim 1, including the following:

6. The step of determining the correlation is as follows: A step in which the correlation between the input index and the system performance index is updated at regular intervals. A method for predicting future system usage according to claim 1, including the following:

7. The step of determining the correlation is as follows: Depending on the type of system performance indicator, in some cases, the correlation between the input indicator and the system performance indicator may be determined by using the value obtained by integrating the time-series data of the input indicator over time. A method for predicting future system usage according to claim 1, characterized by the above.

8. The aforementioned method for predicting future system usage is: The step of using the calculated system performance indicators to recommend the component configuration at a future point in time by the at least one processor. A method for predicting future system usage according to claim 1, further comprising:

9. The step of calculating the system performance indicators at the aforementioned future point in time is: Calculate the system performance indicators for each component constituting the cluster at the aforementioned future point in time. The stage of recommending the component configuration at the aforementioned future point in time is, For each of the aforementioned components, the number of units to be installed is calculated and recommended based on the system performance indicators at the aforementioned future point in time. A method for predicting future system usage according to claim 8, characterized by the above.

10. The aforementioned method for predicting future system usage is: The steps include: when the at least one processor is given a planned input index for building a new cluster, it calculates a system performance index for each component constituting the new cluster using the planned input index and the correlation; The step of calculating and recommending the required number of each component based on the system performance indicators of each component using the aforementioned at least one processor. A method for predicting future system usage according to claim 1, further comprising:

11. The aforementioned method for predicting future system usage is: The step of providing a point in the future where the calculated system performance index exceeds a threshold range, using at least one of the processors. A method for predicting future system usage according to claim 1, further comprising:

12. A computer program for causing a computer device to execute the future system usage prediction method described in any one of claims 1 to 11.

13. At least one processor implemented to execute instructions readable by a computer device Includes, With the aforementioned at least one processor, First, a predictive model for the input indicator is constructed using time-series data of the input indicator stored during a certain period. Second, using the time-series data of the input indicator and the system performance indicator stored during a certain period, the correlation between the input indicator and the system performance indicator is determined. For at least one future point in time, the predictive model is used to predict the input indicator for that future point in time. Using the predicted input indicators and the correlation, the system performance indicators at the future point in time are calculated. The aforementioned input indicators are selected from among the indicators to which system resources respond, in which the change over a third fixed period is less than a threshold. The aforementioned input metric is user traffic corresponding to log data ingested from an external source. The aforementioned system performance indicator is either CPU usage or disk usage, and the computer device is such that

14. With the aforementioned at least one processor, The prediction model is updated using data stored after the previous prediction point for the input indicator. The correlation between the input indicator and the system performance indicator is updated at regular intervals. The computer device according to claim 13, characterized by the above.

15. With the aforementioned at least one processor, Depending on the type of system performance indicator, in some cases, the correlation between the input indicator and the system performance indicator may be determined by using the value obtained by integrating the time-series data of the input indicator over time. The computer device according to claim 13, characterized by the above.

16. With the aforementioned at least one processor, Calculate the system performance indicators for each component constituting the cluster at the aforementioned future point in time. For each of the aforementioned components, the number of units to be installed is calculated and recommended based on the system performance indicators at the aforementioned future point in time. The computer device according to claim 13, characterized by the above.

17. With the aforementioned at least one processor, To provide a point in the future where the calculated system performance indicator exceeds a threshold range. The computer device according to claim 13, characterized by the above.

Images