Time-series prediction management apparatus, time-series prediction management method, and time-series prediction management program

Abstract

To provide a time-series prediction management apparatus, a time-series prediction management method, and a time-series prediction management program capable of determining an anomaly of time-series data when a future predicted value exceeds a reference value, and displaying, on an anomaly determination result screen, a past actual value, the future predicted value, a confidence interval of the predicted value, and the reference value at the same time.SOLUTION: A time-series prediction management apparatus is configured to: generate a time-series prediction model, which is a machine learning model, the model using part of time-series data as training data; obtain predicted values of the time-series data using the time-series prediction model; obtain a confidence interval for the predicted values, on the basis of an anomaly determination definition master; and display a reference value of the time-series data, predicted values, and the confidence interval in time series.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a time series prediction management device, a time series prediction management method, and a time series prediction management program. [Background technology]

[0002] Patent Document 1 discloses a configuration in which, in a failure prediction model for detecting anomalies in industrial machinery, measurement data including time series data relating to control when the industrial machinery is made to perform a certain operation is acquired, multiple time series data are overlaid and displayed in a graph so that they can be confirmed, the time series data is divided into training data and validation data, and the optimal values ​​of machine learning hyperparameters are searched for by Bayesian optimization using the data, training of a failure prediction model is performed based on the training data, the failure prediction model is applied to the validation data, and the degree of anomaly is calculated. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-191680 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the invention described in Patent Document 1 above, it was not possible to display an indicator showing the certainty of the predicted value of future data in the display result of the abnormality judgment, so there was a problem in that it was unclear with what degree of confidence the predicted value could be trusted.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a time series prediction management device, a time series prediction management method, and a time series prediction management program that, in determining an anomaly in time series data, can determine that an anomaly exists when a future predicted value exceeds a reference value, and can simultaneously display the actual value, the future predicted value, the confidence interval of the predicted value, and the reference value on an anomaly determination result screen. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the time series prediction management device of the present invention is a time series prediction management device including a memory unit and a control unit, wherein the memory unit includes a time series storage means for storing time series data set with actual values ​​that depend on time, and an anomaly judgment definition master in which a reference value of the time series data, as well as a reliability and tolerance range of a predicted value of the time series data, are set, and the control unit includes a model creation means for creating a time series prediction model, which is a machine learning model, using a part of the time series data as training data, a time series prediction means for obtaining the predicted value of the time series data using the time series prediction model and obtaining a confidence interval for the predicted value based on the anomaly judgment definition master, and a result display means for displaying the reference value, the predicted value, and the confidence interval of the time series data along a time series.

[0007] In addition, in the time series prediction management device of the present invention, the model creation means divides the time series data into training data, verification data, and test data, and creates the time series prediction model, which is the machine learning model, using the training data.

[0008] In addition, in the time series prediction management device of the present invention, the control unit is further characterized by including a hyperparameter determination means for determining highly adaptable hyperparameters of the time series prediction model by comparing the verification data with the predicted value of the verification data.

[0009] In addition, in the time series prediction management device according to the present invention, the control unit is characterized by further comprising a model suitability evaluation means for obtaining a model suitability evaluation indicating that the suitability of the time series prediction model is sufficient when a proportion of the predicted values ​​of the test data included in the confidence interval is equal to or greater than a predetermined proportion.

[0010] In addition, in the time series prediction management device according to the present invention, the control unit is characterized by further comprising an uncertainty tolerance evaluation means for obtaining an uncertainty tolerance evaluation indicating that the uncertainty of the predicted value is tolerable when the width of the confidence interval is narrower than twice the tolerance width.

[0011] Moreover, a time series prediction management method according to the present invention is a time series prediction management method to be executed by a time series prediction management device having a memory unit and a control unit, the memory unit comprising a time series storage means for storing time series data set with actual values ​​dependent on time, and an anomaly determination definition master in which a reference value of the time series data, and a reliability and tolerance range of a predicted value of the time series data are set, and the time series prediction management method includes a model creation step executed in the control unit for creating a time series prediction model, which is a machine learning model, using a part of the time series data as training data, a time series prediction step for obtaining the predicted value of the time series data using the time series prediction model and obtaining a confidence interval for the predicted value based on the anomaly determination definition master, and a result display step for displaying the reference value, the predicted value, and the confidence interval of the time series data along the time series.

[0012] Furthermore, a time series prediction management program according to the present invention is a time series prediction management program to be executed by a time series prediction management device having a memory unit and a control unit, wherein the memory unit comprises a time series storage means for storing time series data in which actual values ​​dependent on time are set, and an anomaly determination definition master in which a reference value of the time series data, and a reliability and tolerance range of a predicted value of the time series data are set, and the control unit executes a model creation step of creating a time series prediction model, which is a machine learning model, using a part of the time series data as training data, a time series prediction step of obtaining the predicted value of the time series data using the time series prediction model and obtaining a confidence interval for the predicted value based on the anomaly determination definition master, and a result display step of displaying the reference value, the predicted value, and the confidence interval of the time series data along the time series. Effect of the Invention

[0013] According to the present invention, since the results of anomaly detection of time-series data can be visualized on a screen in an easy-to-understand manner, even a user without expert knowledge of machine learning can easily understand the anomaly determination result. In addition, according to the present invention, the effects of preventing corporate fraud, preventing mistakes, and improving business efficiency by detecting changes in business data are achieved. In addition, according to the present invention, the effects of dividing time-series data, determining hyperparameters, evaluating the suitability of a prediction model, and evaluating the tolerance of uncertainty of a predicted value can be executed as a series of processes are achieved. In addition, according to the present invention, since the results of anomaly detection of time-series data can be visualized in an easy-to-understand manner, the effects of being utilized for strengthening internal control and management are achieved. In addition, according to the present invention, since anomaly determination can be performed after a series of evaluation processing steps of a prediction model, the effects of obtaining an anomaly determination result with high reliability are achieved. In addition, according to the present invention, since the actual value of the time-series data, the future predicted value, the confidence interval of the predicted value, and the reference value can be displayed on the display screen of the anomaly determination result, the effects of being able to determine the possibility of exceeding the reference value while grasping the reliability of the predicted value are achieved. In addition, the present invention has an advantage that a series of model evaluation processes can be performed to obtain a reliable prediction result, and that the risk of a normal state being erroneously determined to be abnormal can be reduced. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing an example of trend data of an employee's overtime hours. [Diagram 2] FIG. 2 is a diagram showing an example of trend data of an employee's overtime hours. [Diagram 3] FIG. 3 is a diagram showing an example of a prediction function of the standardized sum of squares error in this embodiment. [Figure 4] FIG. 4 is a diagram showing an example of an abnormality determination result in this embodiment. [Diagram 5]FIG. 5 is a block diagram showing an example of the configuration of a time-series prediction management device in this embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the process of the time-series prediction management device in this embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 18] FIG. 18 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 20] FIG. 20 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 21]FIG. 21 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 22] FIG. 22 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 23] FIG. 23 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 24] FIG. 24 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 25] FIG. 25 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 26] FIG. 26 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 27] FIG. 27 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 28] FIG. 28 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 29] FIG. 29 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 30] FIG. 30 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 31] FIG. 31 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 32] FIG. 32 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 33] FIG. 33 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 34] FIG. 34 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 35] FIG. 35 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 36] FIG. 36 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 37] FIG. 37 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 38] FIG. 38 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Figure 39] FIG. 39 is a diagram illustrating an example of the time-series prediction management process in this embodiment. [Diagram 40] FIG. 40 is a diagram showing an example of information to be passed to the time series prediction component in this embodiment. [Diagram 41] FIG. 41 is a diagram showing an example of a decomposition result in this embodiment. [Diagram 42] FIG. 42 is a diagram showing an example of a prediction result in this embodiment. [Diagram 43] FIG. 43 is a diagram showing an example of pre-processing in this embodiment. [Diagram 44] FIG. 44 is a diagram showing an example of pre-processing in this embodiment. [Diagram 45] FIG. 45 is a diagram showing an example of pre-processing in this embodiment. [Figure 46] FIG. 46 is a diagram showing an example of a prior distribution in this embodiment. [Figure 47] FIG. 47 is a diagram showing an example of a prior distribution in this embodiment. [Figure 48] FIG. 48 is a diagram showing an example of a prior distribution in this embodiment. [Figure 49] FIG. 49 is a diagram showing an example of a prediction function in this embodiment. [Figure 50] FIG. 50 is a diagram showing an example of a prediction function in this embodiment. [Figure 51] FIG. 51 is a diagram showing an example of a prediction function in this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the embodiment.

[0016] [1. Overview] First, an overview of the present invention will be described with reference to Fig. 1 to Fig. 4. Fig. 1 and Fig. 2 are diagrams showing an example of trend data of an employee's overtime hours. Fig. 3 is a diagram showing an example of a prediction function of a standardized sum of squares error in this embodiment. Fig. 4 is a diagram showing an example of an abnormality determination result in this embodiment.

[0017] Traditionally, companies that manage time-series data with trends and periodicity have had the demand to predict future data based on past time-series data and make decisions based on the prediction results. For example, traditionally, (a) companies that manage working hours have had the demand to "detect in advance employees who may exceed the legally prescribed overtime hours" and "detect in advance organizations where the total overtime hours of employees in each organization is likely to exceed a certain specified value" by using working data, and (b) companies that manage hardware have had the demand to "detect in advance hardware that is likely to exceed the specified values ​​of hardware resource usage, server CPU usage, memory usage, hard disk access count, etc." by using hardware monitoring data. In addition to the demands for (a) the utilization of working data and (b) the utilization of hardware monitoring data, similar demands have been made for all business data that have trends and periodicity.

[0018] That is, in the past, (a) there was a demand in companies that manage working hours to "pre-detect employees who may exceed the legally prescribed overtime hours," and the data to be analyzed was trend data of the non-legal hours counter for each working day, as shown in the graph in Figure 1. Here, the time series data of the non-legal hours counter for each working day had the following characteristics: feature (1): the non-legal hours counter is reset to 0 hours on the attendance closing date (the 15th of each month in Figure 1); feature (2): the non-legal hours counter tends to increase monotonically from the day after the attendance closing date to the next closing date (closing period), and has periodicity during the closing period; and feature (3): the slope and maximum value of the increase in the non-legal hours counter differ for each closing period. Here, the non-legal hours counter may be the cumulative total of non-legal overtime hours from the previous attendance closing date to a certain point in time.

[0019] As shown in Figure 1, conventionally, non-statutory hours have characteristics such as characteristics (1), (2), and (3). Therefore, there was a demand for "predicting the increasing trend of non-statutory hours for a future period from past performance, since the maximum value of non-statutory hours is different for each closing period, and for detecting in advance with high accuracy as an anomaly employees who are likely to exceed the legally stipulated monthly upper limit."

[0020] As shown in Figure 2, it has traditionally been possible to perform machine learning based on past performance data to obtain predicted values ​​for future data as well as upper and lower confidence limits for the confidence interval.

[0021] However, in the past, there was a problem (1) that it took a long time to search for hyperparameters of a prediction model. That is, when creating a machine learning model using the conventional technology, there are hyperparameters that need to be set by the user, and the suitability of the prediction model changes significantly depending on the setting value. However, it is difficult for anyone other than an AI engineer such as a data scientist to find appropriate values ​​for these hyperparameters, and manual search takes a lot of time. Therefore, as a conventional method for a system to automatically find hyperparameters with good accuracy so that anyone can use it flexibly, there was a method of setting all possible candidate values ​​for the hyperparameters to obtain the most accurate hyperparameters. However, this method requires a huge amount of trial calculation when there are many candidate values, which is practically impossible, and the problem of processing speed becomes more pronounced because a machine learning model is created for each time series data to be predicted. Here, the hyperparameters are parameters for controlling the learning process and prediction behavior of the model in machine learning, and the time series data is data having a time sequence and a value sequence that depends on the time, and the time sequence may be a sequence representing the time of the time series data.

[0022] In addition, conventional systems have had problems with false positives in anomaly judgments and the uncertainty of the predicted values ​​used to judge anomalies (2). That is, conventional anomaly detection systems only detect when a predicted value exceeds a reference value and judge an anomaly to be occurring, so erroneous anomaly judgments are likely to occur even when the certainty of the prediction results for future data is low, and since an index showing the certainty of the predicted values ​​for future data was not displayed in the display results of anomaly judgments, it was unclear how certain the predicted values ​​could be.

[0023] Therefore, in this embodiment, a solution (1) is adopted for problem (1) in which optimal hyperparameters are searched for using Bayesian optimization. Here, singular spectrum analysis (SSA) is used to perform time series prediction, but SSA has the characteristics of being able to learn trends and periodicity and being resistant to noise. In order to improve the suitability of a prediction model using SSA, it is important to effectively extract trend and seasonal components from time series data and remove noise (smoothing effect). In addition, in the past, in order to perform SSA under good conditions, it was necessary to set a window size, which is an appropriate hyperparameter to be searched for using Bayesian optimization. Although the range of values ​​that the window size can take is known, a method for efficiently finding the optimal window size has not been established, and the optimal value was found by a brute force method of trying all possible values ​​for the window size, which took time. Therefore, in this embodiment, when searching for hyperparameters, Bayesian optimization suitable for the purpose is used to search for hyperparameters that minimize the prediction error (minimum value search).

[0024] That is, in this embodiment, a mechanism is provided that improves the suitability of a time series prediction model and shortens the search time by skillfully applying Bayesian optimization to the search for optimal hyperparameters. Specifically, in this embodiment, in the processing step of performing Bayesian optimization, the processing time limit for Bayesian optimization is determined according to the processing time limit specified by the user, and by skillfully applying Bayesian optimization, it is possible to search for hyperparameters that improve the suitability of a prediction model to a certain extent within the time limit, so that time series data can be analyzed by the user's specification of the processing time limit. In addition, in this embodiment, as the initial observation points, the minimum value, the third division point, and the maximum value of the hyperparameter to be searched are observed, and as the fifth point and thereafter, the number of observation points is increased according to the processing time per observation point within the time limit, thereby securing as many observation points as possible within the time limit. Note that by increasing the number of observation points, more optimal hyperparameters can be obtained, but at the same time, the processing time is increased. However, in this embodiment, the balance is adjusted so that the optimal hyperparameters can be obtained to a certain extent within the time limit by utilizing Bayesian optimization. Furthermore, when performing Bayesian optimization, it is necessary to specify a kernel function. There are many types of kernel functions, but in this embodiment, a Gaussian kernel is specified so as to be able to flexibly respond to any function to be searched. When specifying the Gaussian kernel, it is also necessary to specify the coefficients of the Gaussian kernel, and a method for determining the optimal coefficients is explored by conducting experiments using a plurality of coefficients.

[0025] Here, FIG. 3 shows the predicted value and prediction interval of the prediction error (sum of squares error) standardized for the window size when nine hyperparameters are searched. There are 53 settable hyperparameters, but for the nine searched points, the search is centered on hyperparameters whose standardized sum of squares error is a negative value, so the search is performed with the goal of finding hyperparameters that minimize the standardized sum of squares error. In addition, in this embodiment, by using Bayesian optimization, it is possible to find hyperparameters that minimize the prediction error even with a small number of trial calculations. Note that, as shown in FIG. 3, in this embodiment, the further away from the observation point is, the lower the certainty of the predicted value returned by the prediction function, so the uncertainty increases and the range represented by "predicted value ± standard deviation" becomes wider.

[0026] In addition, in this embodiment, a solution (2) is adopted for the problem (2), in which a predicted value is judged to be abnormal when it exceeds a reference value, and the abnormality judgment result is displayed together with a confidence interval according to the reliability specified by the user. Specifically, in this embodiment, in the division of time series data, the time series data for the actual period is divided into three, data for training, data for verification, and data for testing, and in order to build a model that can perform highly reliable predictions for future data, optimal hyperparameters are determined in the "hyperparameter determination" process and the "evaluation of the suitability of the prediction model" process, and the reliability of the predicted value is evaluated with a tolerance range for the future data prediction result in the "evaluation of the tolerance of the uncertainty of the predicted value" process. In this way, in this embodiment, an abnormality judgment is performed after a series of evaluation processes, so that an abnormality judgment result with more reliability can be obtained. In addition, in this embodiment, when the abnormality judgment result that finally satisfies a certain tolerance is finally displayed, the predicted value, the confidence interval of the predicted value that shows the tolerance of the uncertainty of the prediction, and the reference value are displayed, and in particular, the confidence interval that is the tolerance of the uncertainty of the predicted value is displayed, so that the user can determine whether or not there is a possibility that the reference value will be exceeded while grasping the reliability of the predicted value.

[0027] Here, as shown in FIG. 4, in this embodiment, the actual value of the time series data to be predicted, the future predicted value, the confidence interval of the predicted value (the reliability of the confidence interval can be specified by the user), and the reference value are displayed as the abnormality judgment result, and it is judged that the abnormality is caused when the predicted value exceeds the reference value as the abnormality judgment. However, when the confidence interval has a width of "predicted value ± tolerance width" or more, the certainty of the predicted value is low, and the abnormality judgment is not performed, so that the reliability of the predicted value and the abnormality judgment result are guaranteed. That is, as shown in FIG. 4, in this embodiment, various data are displayed in the abnormality judgment result, and the user can compare it with the actual data through the graph display of the abnormality judgment result, and by checking the confidence interval specified by the user and the difference between the trend of the predicted value and the reference value, it is possible to grasp whether there is a possibility that the reference value will be exceeded, and as a result, appropriate countermeasures can be taken. Also, as shown in FIG. 4, in this embodiment, it is possible to predict from the abnormality judgment result that the non-statutory time system is likely to exceed the reference value of 45 hours as of May 9, 2023, and it is possible to alert employees who have been judged to be abnormal in advance.

[0028] [2. Configuration] An example of the configuration of the time series prediction management device 100 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the configuration of the time series prediction management device 100 in this embodiment.

[0029] 5, the time series prediction management device 100 is a commercially available desktop personal computer. Note that the time series prediction management device 100 is not limited to a stationary information processing device such as a desktop personal computer, but may be a portable information processing device such as a commercially available notebook personal computer, a PDA (Personal Digital Assistant), a smartphone, or a tablet personal computer.

[0030] The time series prediction management device 100 includes a control unit 102, a communication interface unit 104, a storage unit 106, and an input / output interface unit 108. The units included in the time series prediction management device 100 are connected to each other so as to be able to communicate with each other via any communication path.

[0031] The communication interface unit 104 communicatively connects the time series prediction management device 100 to a network 300 via a communication device such as a router and a wired or wireless communication line such as a dedicated line. The communication interface unit 104 has a function of communicating data with other devices via the communication line. Here, the network 300 has a function of connecting the time series prediction management device 100 and the server 200 to each other so that they can communicate with each other, and is, for example, the Internet or a LAN (Local Area Network).

[0032] An input device 112 and an output device 114 are connected to the input / output interface unit 108. The output device 114 may be a monitor (including a touch panel), a speaker, or a printer. The input device 112 may be a keyboard, a mouse, a microphone, or a monitor that cooperates with a mouse to realize a pointing device function. In the following, the output device 114 may be referred to as the monitor 114 or the printer 114, and the input device 112 may be referred to as the keyboard 112 or the mouse 112.

[0033] The storage unit 106 stores various databases, tables, files, and the like. The storage unit 106 records computer programs for cooperating with an operating system (OS) to give instructions to a central processing unit (CPU) to perform various processes. For example, memory devices such as random access memory (RAM) and read only memory (ROM), fixed disk devices such as hard disks, flexible disks, and optical disks can be used as the storage unit 106. The storage unit 106 includes a time series database 106a and an abnormality determination definition master 106b.

[0034] The time-series database 106a stores time-series data. Here, the time-series database 106a may store time-series data in which a performance value dependent on time is set. The time-series database 106a may also store training data, verification data, test data, anomaly determination result data, a time-series prediction model, a predicted value of the time-series data, hyperparameters, a model suitability evaluation, an uncertainty tolerance evaluation, anomaly determination results, update data, and business data.

[0035] The abnormality determination definition master 106b is a master that sets an abnormality determination definition on business data. Here, the abnormality determination definition master 106b may set a processing allowable time for searching for hyperparameters. Furthermore, the abnormality determination definition master 106b may set a reference value of time series data, and a reliability and allowable range of a predicted value of time series data. Furthermore, the abnormality determination definition master 106b may set a parameter setting value of time series data.

[0036] The control unit 102 is a CPU or the like that performs overall control of the time series prediction management device 100. The control unit 102 has an internal memory for storing control programs such as an OS, programs that define various processing procedures, required data, and the like, and executes various information processing based on these stored programs. The control unit 102 conceptually includes a model creation unit 102a, a time series prediction unit 102b, a hyperparameter determination unit 102c, a model suitability evaluation unit 102d, an uncertainty tolerance evaluation unit 102e, an anomaly detection unit 102f, and a result display unit 102g.

[0037] The model creation unit 102a creates a time series prediction model that is a machine learning model. Here, the model creation unit 102a may create a time series prediction model that is a machine learning model by using a part of the time series data as training data. The model creation unit 102a may create a time series prediction model that is a machine learning model by using a part of the time series data as training data using Bayesian optimization within a time limit determined according to the processing allowable time. The model creation unit 102a may further obtain four points, the minimum value, the third division point, and the maximum value, of the hyperparameter as initial observation points of the hyperparameter, and create a time series prediction model with each initial observation point as a hyperparameter based on the training data. The model creation unit 102a may calculate a standardized sum of squares error from the sum of squares error within the time limit, obtain candidate observation points of the hyperparameter that can be set as much as possible within the time limit using Bayesian optimization based on the standardized sum of squares error, and create a time series prediction model with each candidate observation point as a hyperparameter based on the training data. The model creation unit 102a may determine a coefficient of a Gaussian kernel capable of evaluating the similarity between observation points, calculate a standardized sum of squares error from the sum of squares error within a time limit, and obtain candidate observation points of hyperparameters that can be set as much as possible within a time limit using Bayesian optimization based on the standardized sum of squares error and the Gaussian kernel with the coefficient set, and create a time series prediction model with each candidate observation point as a hyperparameter based on training data. Here, the coefficient of the Gaussian kernel may be 5 when the quotient obtained by dividing the number of candidate points to be searched by 5 is equal to or less than the number of candidate observation points, and may be the quotient obtained by dividing the number of candidate points to be searched by the number of candidate observation points when the quotient is greater than the number of candidate observation points. The model creation unit 102a may also divide the time series data into training data, verification data, and test data, and create a time series prediction model that is a machine learning model using the training data.

[0038] The time series prediction unit 102b obtains a predicted value of the time series data. Here, the time series prediction unit 102b may obtain the predicted value of the time series data using a time series prediction model. Also, the time series prediction unit 102b may obtain a predicted value of the time series data using the time series prediction model, and obtain a confidence interval for the predicted value based on the anomaly determination definition master 106b. Also, the time series prediction unit 102b may obtain a predicted value of the verification data and the test data using the time series prediction model, and obtain a confidence interval for the predicted value based on the anomaly determination definition master 106b.

[0039] The hyperparameter determination unit 102c determines a hyperparameter having high adaptability for the time series prediction model. Here, the hyperparameter determination unit 102c may determine the hyperparameter having high adaptability for the time series prediction model based on the actual value and the predicted value. The hyperparameter determination unit 102c may also calculate the square sum error between the actual value and the predicted value, and determine the observation point having the smallest square sum error as the hyperparameter having high adaptability for the time series prediction model. The hyperparameter determination unit 102c may also determine the hyperparameter having high adaptability for the time series prediction model by comparing the verification data with the predicted value of the verification data.

[0040] The model suitability evaluation unit 102d obtains a model suitability evaluation indicating an evaluation of the suitability of the time series prediction model. Here, the model suitability evaluation unit 102d may obtain a model suitability evaluation indicating that the suitability of the time series prediction model is sufficient when the proportion of the predicted values ​​of the test data included in the confidence interval is equal to or greater than a predetermined proportion.

[0041] The uncertainty tolerance evaluation unit 102e acquires an uncertainty tolerance evaluation indicating an evaluation of the tolerance of the uncertainty of the predicted value. Here, the uncertainty tolerance evaluation unit 102e may acquire an uncertainty tolerance evaluation indicating that the uncertainty of the predicted value is tolerable. Furthermore, when the width of the confidence interval is narrower than twice the tolerance width, the uncertainty tolerance evaluation unit 102e may acquire an uncertainty tolerance evaluation indicating that the uncertainty of the predicted value is tolerable. The uncertainty tolerance evaluation unit 102e may acquire an uncertainty tolerance evaluation indicating that the uncertainty of the predicted value is not tolerable.

[0042] The anomaly detection unit 102f acquires an anomaly determination result of the predicted value. Here, when the anomaly detection unit 102f determines that the predicted value exceeds a reference value, the anomaly detection unit 102f may acquire an anomaly determination result of the predicted value.

[0043] The result display unit 102g displays the prediction result of the time series data. Here, the result display unit 102g may display the reference value, the predicted value, and the confidence interval of the time series data in time series. The result display unit 102g may also display the abnormality determination result of the predicted value.

[0044] [3. Specific Examples] A specific example of this embodiment will be described with reference to FIGS.

[0045] [Time series prediction management processing] An example of a time series prediction management process in this embodiment will now be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of a process of the time series prediction management device 100 in this embodiment.

[0046] As shown in FIG. 6, the model creation unit 102a divides the time-series data into training data, verification data, and test data (step SA-1).

[0047] Then, the model creation unit 102a creates a time-series prediction model, which is a machine learning model, using the training data (step SA-2).

[0048] Then, the time series prediction unit 102b obtains predicted values ​​of the verification data and the test data using the time series prediction model, and obtains a confidence interval for the predicted values ​​based on the anomaly determination definition master 106b (step SA-3).

[0049] Then, the hyperparameter determination unit 102c determines hyperparameters with high adaptability for the time series prediction model by comparing the validation data with the predicted value of the validation data (step SA-4).

[0050] Then, the model suitability evaluation unit 102d acquires a model suitability evaluation indicating an evaluation of the suitability of the time series prediction model (step SA-5).

[0051] Then, the uncertainty tolerance evaluation unit 102e acquires an uncertainty tolerance evaluation indicating an evaluation of the tolerance of the uncertainty of the predicted value (step SA-6).

[0052] Then, the abnormality detection unit 102f determines whether or not the predicted value exceeds a reference value (step SA-7).

[0053] If the abnormality detection unit 102f determines that the predicted value exceeds the reference value (step SA-7: Yes), the abnormality detection unit 102f shifts the process to step SA-8.

[0054] Then, the abnormality detection unit 102f acquires the abnormality determination result of the predicted value (step SA-8).

[0055] Then, the result display unit 102g causes the output device 114 to display the abnormality judgment result of the predicted value, and causes the output device 114 to display the reference value, the predicted value, and the confidence interval of the time series data in chronological order (step SA-9), thereby completing the processing.

[0056] On the other hand, if the abnormality detection unit 102f determines that the predicted value does not exceed the reference value (step SA-7: No), it shifts the process to step SA-10.

[0057] Then, the result display unit 102g causes the output device 114 to display the reference values, predicted values, and confidence intervals of the time-series data in chronological order (step SA-10), and the process ends.

[0058] Here, an example of the time series prediction management process in this embodiment will be described with reference to Fig. 7 to Fig. 39. Fig. 7 to Fig. 39 are diagrams showing an example of the time series prediction management process in this embodiment.

[0059] In this embodiment, anomaly detection processing is performed every night at a company whose attendance records are closed on the 15th of each month. When the abnormality detection date is 5 / 2 / 2023, and actual overtime data for employees up to 5 / 1 / 2023 is already available, the purpose is to detect in advance employees who are likely to have their non-statutory hours exceed 45 hours by the next closing date, 5 / 15 / 2023.

[0060] First, as shown in FIG. 7, in this embodiment, it is assumed that the setting for executing the abnormality judgment is registered in the abnormality judgment definition master 106b that manages the definition for performing the abnormality judgment on the business data (1). Here, the reliability is not a required item, and is set only when it is desired to evaluate the width of the confidence interval. The tolerance is not a required item, and is set only when it is desired to evaluate the width of the confidence interval. The tolerance sets the width of the range in which the deviation between the predicted value and the actual value is allowed. The prediction period end value acquisition function is a parameter that sets how far into the future the time series item is to predict the time series data. fc_GetNextKintaiSimebi is a scalar value function that can be used in the database, and by executing this function, the next attendance closing date based on the execution date is obtained. In this embodiment, since the execution date is 2023 / 5 / 2, the date 2023 / 5 / 15, which is the 15th day, is obtained by executing this function. The processing allowable time is set to the time that may be taken from "preprocessing" to "abnormality judgment processing". In addition, the process from "pre-processing" to "abnormality determination process" is performed for each data group divided by the prediction unit, and the processing time allowed until the processing is completed for all data groups is set. For example, in this embodiment, since the prediction unit is employee CD, the process from "pre-processing" to "abnormality determination process" is performed for each employee CD, that is, for each employee, and the allowable processing time required for the entire process for the number of employees is set to 20 minutes.

[0061] In addition, as shown in Fig. 8, this embodiment is based on the premise (2) that actual overtime data for each employee already exists. Here, as shown in Fig. 8, in the actual overtime data, the day of the week column is not used for calculation, but since data for "Saturday" and "Sunday" are interpolated in preprocessing, in the case of a company where employees work on weekdays, data without "Saturday" and "Sunday" is used.

[0062] Also, as shown in Fig. 9, in this embodiment, it is assumed (3) that a component that predicts time series data (time series prediction component) already exists. Here, as shown in Fig. 9, various information is passed to the time series prediction component, and a prediction result is returned. Also, as shown in Fig. 9, the reliability is not a required item, and is set only when it is desired to evaluate the width of the confidence interval. If it is not set, the information returned by the time series prediction component does not include an upper confidence limit and a lower confidence limit.

[0063] Also, as shown in FIG. 10, in this embodiment, it is assumed (4) that a component that performs Bayesian optimization (Bayesian optimization component) is already present. Here, as shown in FIG. 10, the Bayesian optimization component receives various information and returns the next point to be searched. Also, as shown in FIG. 10, the Gaussian kernel (ρ=5) is a function in which 5 is inserted into the coefficient ρ in the Gaussian kernel formula: k(x1, x2)=exp[-|x1-x2| 2 / 2ρ 2 ], and functions are passed to the kernel information in this way. Also, as shown in Figure 10, the Bayesian optimization component has a function for retaining information passed to it, and information passed to the Bayesian optimization component during a search is retained in a structure similar to "retained information." Also, as shown in Figure 10, each time information is passed to the Bayesian optimization component during a search, a row is added to "retained information."

[0064] As shown in FIG. 11, in this embodiment, the pre-processing is performed for the purpose of facilitating the analysis of time series data. Usually, overtime data, etc., lacks data on Saturdays, Sundays, and holidays, and the time sequence is not equally spaced, making analysis difficult. Therefore, pre-processing is performed when the time sequence items are not equally spaced. That is, as shown in FIG. 11, in this embodiment, the time series data is processed so that the time sequence items are equally spaced, and if the time sequence items are already equally spaced, this processing may be skipped. As a result, in this embodiment, the time series data is processed so that the time sequence is equally spaced (in the case of overtime data, Saturday and Sunday data is interpolated), and data with an equally spaced time sequence is created. Specifically, as shown in FIG. 11, in this embodiment, processing is performed based on the data registered in the parameter setting column of the abnormality determination definition master 106b (pre-processing is performed so that the time sequence items of the parameter setting are equally spaced, and when performing interpolation, etc., interpolation is performed based on the value of the column of the prediction item of the parameter setting). Furthermore, as shown in Figure 11, the processed data has the following characteristics: the time column items are equally spaced from row to row, the data for "Saturday" and "Sunday" are interpolated in the day of the week column, and the workday column is equally spaced with a difference of one day between each row.

[0065] As shown in FIG. 12, the present embodiment aims to create time series data having a time sequence for the period to be predicted. That is, as shown in FIG. 12, in the present embodiment, time series data having a time sequence for the period to be predicted is created. As a result, in the present embodiment, information on "how many points to predict" required for constructing a prediction model is obtained by creating time series data for the period to be predicted in this step, and when the abnormality determination result is displayed, information on "when in the future the reference value will be exceeded" is displayed, and this information is required to represent that time point. Specifically, as shown in FIG. 12, in the present embodiment, (1) the time interval of the time sequence item is calculated, and (2) time series data having a time sequence for the period to be predicted is created based on the prediction period end value acquisition function set in the abnormality determination definition master 106b. Also, as shown in FIG. 12, in the present embodiment, since the abnormality determination process is executed on 2023 / 5 / 2, when the prediction period end value acquisition function is executed, 2023 / 5 / 15, which is the next attendance closing date after the execution date, is acquired as the prediction period end value.

[0066] Then, as shown in Fig. 13, in this embodiment, the time interval calculated in Fig. 12(1) is added to the working date of the final record of the pre-processed overtime actual data until it reaches the prediction period end value of the parameter setting, and time series data having a time sequence of the period to be predicted is created. That is, as shown in Fig. 13, in this embodiment, the time interval (1 day) is added until it reaches the prediction period end value. Also, as shown in Fig. 13, in the time series data, the non-legal hours column is set to NULL because it is future data.

[0067] As shown in FIG. 14, the present embodiment aims to prepare data for "determining hyperparameters" and "evaluating the suitability of a prediction model". That is, as shown in FIG. 14, in the present embodiment, the time series data of the preprocessed actual period is divided into training data, validation data, and test data. Specifically, as shown in FIG. 14, in the present embodiment, when the time series data of the preprocessed actual period is arranged in ascending order of time sequence, 80% of the total data from the past time is set as training data, 10% as validation data, and the remaining as test data. Here, the ratio of validation data to test data is generally set to 10% to 20%, respectively, but even in the case of time series data having a long period, the ratio of training data, validation data, and test data is set to 80%:10%:10%, with an emphasis on increasing the ratio of training data so that periodicity can be learned. In the present embodiment, if the characteristics of the time series data are known, the division ratio may be changed to another ratio such as 60%:20%:20%.

[0068] As shown in FIG. 15, in this embodiment, when singular spectrum analysis (SSA) is used in time series prediction, the purpose is to efficiently search for an optimal window size (hyperparameter in time series prediction component). That is, as shown in FIG. 15, in this embodiment, there is a hyperparameter called window size that the user must specify in time series prediction, and the purpose is to efficiently process in a short time using a time limit, initial observation point, etc. by predicting validation data with training data for various window sizes and searching for an optimal window size by performing Bayesian optimization based on the prediction result. As a result, in this embodiment, the optimal window size is searched for, improving the suitability of the prediction model using SSA, making it possible to accurately extract trends and seasonal components and remove noise, and by using Bayesian optimization, the optimal window size can be efficiently searched for, thereby significantly shortening the search time compared to a brute force method. In addition, in this embodiment, the point is to achieve both improvement in the suitability of the time series prediction model and shortening the search time by skillfully applying Bayesian optimization to the search for the optimal hyperparameter, which is a useful approach to optimizing the window size of SSA. Here, "achieving both improved suitability of the time series prediction model and reduced search time by skillfully applying Bayesian optimization" means evaluating the error by comparing the prediction results of the validation data (actual results) with the validation data when the window size is changed, using the error information for window size search by Bayesian optimization, setting a time limit, and finding the window size with the smallest error within the range of the window size search found within the time, thereby achieving both reduced search time and improved accuracy.

[0069] Specifically, as shown in Fig. 16, in this embodiment, "1. Allowable processing time for parameter setting of abnormality determination definition master 106b" and "2. Number of data groups divided for each prediction unit of parameter setting of abnormality determination definition master 106b" are used to set the search time limit. Here, the number of data groups is the number of data groups that repeat "pre-processing" to "abnormality determination processing" in the processing flow, and since the prediction unit is employee CD, it is the same as the number of employee CDs. In addition, the search time limit is calculated and set using the allowable processing time and the number of data groups, using the formula "search time limit = allowable processing time ÷ number of data groups".

[0070] Then, as shown in FIG. 17, in this embodiment, an optimal value of the window size, which is a hyperparameter to be passed to the time series prediction component, is searched for. Here, the window size, which is a hyperparameter to be passed to the time series prediction component, needs to be 2 or more and less than half the number of data in the training data. Also, as shown in FIG. 17, in this embodiment, the maximum value, minimum value, and third division point values ​​are extracted from the hyperparameter candidates (Latin Hypercube method) and are set as initial observation points. That is, in this embodiment, the maximum value, minimum value, and third division point values ​​are extracted from the hyperparameter candidates and are set as initial observation points, thereby covering a wide area in the initial stage of Bayesian optimization and reducing the risk of bias toward a local optimal solution.

[0071] Then, as shown in FIG. 18, in this embodiment, for each initial observation point, (3)-1: prediction of validation data, and (3)-2: receiving information returned from the time series prediction component and calculating the sum of squared errors by comparing with the validation data are performed, and the validation results are obtained as the state after the processes of (3)-1 and (3)-2 are performed for the four initial observation points.

[0072] As shown in FIG. 19, in this embodiment, standardization is performed so that the average becomes 0 in Bayesian optimization, and learning proceeds efficiently. Therefore, the average value and standard deviation used in the standardization process of the sum of squares error are retained for use in subsequent processing.

[0073] Then, in this embodiment, information required for Bayesian optimization is passed to the Bayesian optimization component before the search, as shown in Fig. 20. Note that in this embodiment, as shown in Fig. 20, in order to determine the coefficients of the Gaussian kernel, candidates for hyperparameters to be searched and the number of hyperparameters that are likely to be searched within the time limit are required, and the number of hyperparameters that are likely to be searched within the time limit is calculated. Here, the calculation of {the processing time required for (3) and (4) ÷ 4} corresponds to the time required for predicting validation data and standardizing validation data for one hyperparameter.

[0074] Then, in this embodiment, the next candidate point returned from the Bayesian optimization component after the initial observation point is passed is obtained, as shown in Fig. 21. Here, in this embodiment, "information to be held" is obtained from "information (1) and (2) passed during search" as shown in Fig. 21.

[0075] Then, as shown in Fig. 22, in this embodiment, after the process of (6), the next hyperparameter to be searched is obtained, and the information is passed to the time series prediction component. Here, as shown in Fig. 22, in this embodiment, after prediction is performed using the time series prediction component, the square sum error is calculated based on the prediction result obtained from the time series prediction component. Then, as shown in Fig. 22, in this embodiment, the mean and standard deviation at the time of standardization are the mean and standard deviation of the initial observation point standardized information obtained by standardizing the square sum error in the process of (4), and the verification result is stored in addition to the verification result of the initial observation point.

[0076] Then, as shown in FIG. 23, in this embodiment, the observation result is passed to the Bayesian optimization component, and the next point to be searched is obtained from the Bayesian optimization component.

[0077] Then, in this embodiment, after the process of (7) is performed as long as the time limit allows, verification result data of the window size and the standardized square sum error is obtained, as shown in Fig. 24. Here, in this embodiment, verification is performed for a window size of 10 points within the time limit, and the square sum error is smallest when the window size is 54, so prediction is performed using the window size of 54 from the process of (8) onwards.

[0078] Next, as shown in FIG. 25, in this embodiment, the purpose is to confirm whether the window size, which is a hyperparameter determined in the process of "determining hyperparameters (predicting validation data)", is appropriate. That is, as shown in FIG. 25, in this embodiment, the purpose is to confirm whether an appropriate prediction model is constructed by the setting value of the searched window size. As a result, in this embodiment, if the window size determined in the process of "determining hyperparameters" is appropriate, the test data and its predicted value will be close to each other, so that the prediction accuracy of the model can be evaluated. In addition, in this embodiment, the accuracy is evaluated by the proportion of the test data included in the confidence interval of the predicted value. In addition, as shown in FIG. 25, in this embodiment, after the information on the predicted value and the 95% confidence interval is obtained from the time series prediction component, if the proportion of the test data included in the confidence interval is 95% or more, the suitability of the prediction model is evaluated as sufficient. In addition, in this embodiment, the model evaluation is performed using the window size and test data determined in the process of "determining hyperparameters".

[0079] Specifically, in this embodiment, the training and validation data, the optimal window size, the prediction score, and the reliability are passed to the time series prediction component to perform prediction, as shown in Fig. 26. Here, in this embodiment, the training and validation data are combined to form the training and validation data, and a fixed value of 0.95 is set in advance on the system side as the reliability passed to the time series prediction component in the process of "evaluating the suitability of the model."

[0080] As shown in FIG. 27, in this embodiment, the prediction result and confidence interval are compared with the test data to evaluate the suitability of the prediction model. Here, in this embodiment, the evaluation method is such that if the rate (coverage) of the test data included in the 95% confidence interval of the prediction value is 95% or more, the prediction model is evaluated as being appropriate, and the confidence level of 95% for determining the width of this confidence interval is not specified by the user but is set as a fixed value in the system, and when evaluating the suitability of a prediction model using the confidence interval, a confidence interval with a 95% confidence level is generally used, so 95% is adopted. As shown in FIG. 27, in this embodiment, the rate at which the test data is included in the confidence interval is obtained. Here, "test data is included in the confidence interval" means that the test data is above the lower confidence limit and below the upper confidence limit. When the test data is "included in the confidence interval", "circle: conforming" is entered in the inclusion flag column, and when the test data is not included, "X: non-conforming" is set in the inclusion flag column, and the proportion of data included in the confidence interval is the coverage, which is calculated as "coverage = number of circles in the inclusion flag ÷ number of test data × 100 (%)". In this embodiment, since all data are included, the coverage is 100%, and since it is greater than 95%, it is determined that the suitability of the prediction model is sufficient. Conversely, if the coverage is less than 95%, it is determined that the suitability of the prediction model is low, and the "future data prediction" process and the "abnormality determination" process are skipped, and the process moves to the process for the data of the next prediction unit.

[0081] Next, as shown in FIG. 28, in this embodiment, the purpose is to predict future data and evaluate whether the predicted value is reliable. That is, as shown in FIG. 28, in this embodiment, the user can specify the reliability and tolerance range that need to be set when evaluating the reliability of the predicted value, so that the user can decide how much reliability is required for anomaly judgment, and the reliability of the predicted value is evaluated based on the width of the confidence interval and the value of the tolerance range. As a result, in this embodiment, the tolerance of the uncertainty of the predicted value is evaluated based on the width of the confidence interval of the reliability specified by the user, so that it is possible to proceed to anomaly judgment processing based on a prediction result with uncertainty that the user can tolerate.

[0082] Specifically, in this embodiment, the time series data of all post-preprocessing actual periods, the optimal window size, the prediction score, and the reliability are passed to the time series prediction component to perform prediction, as shown in Fig. 28. Here, in this embodiment, the reliability value of the parameter setting in the anomaly determination definition master 106b is passed, and if the reliability is not set in the master table, the reliability value is not passed to the time series prediction component, as shown in Fig. 28.

[0083] Then, as shown in FIG. 29, in this embodiment, the width of the confidence interval is compared with twice the tolerance width to evaluate the tolerance of the uncertainty of the predicted value (if the reliability of the parameter setting is not set in the abnormality determination definition master 106b, the process of "evaluating the tolerance of the uncertainty of the predicted value" is skipped). Here, in this embodiment, the tolerance width is the tolerance width of the parameter setting in the abnormality determination definition master 106b. Also, in this embodiment, the information returned from the time series prediction component includes the upper limit (upper confidence limit) and the lower limit (lower confidence limit) of the confidence interval, and if the deviation of the actual value from the predicted value by the tolerance width is allowed, the width of the confidence interval becomes important when evaluating the certainty of the predicted value. Here, the width of the confidence interval represents the uncertainty of the predicted value, which is the difference between the upper confidence limit and the lower confidence limit, and the wider the width, the higher the uncertainty. Also, in this embodiment, if the width of the confidence interval is twice or more the tolerance width, the predicted value may have a deviation of more than the tolerance width in both the positive and negative directions, and it can be determined that the certainty of the prediction is low. On the other hand, in this embodiment, when the width of the confidence interval is equal to or less than twice the tolerance, the predicted value may fall within the deviation of the tolerance in both the positive and negative directions, and the reliability of the prediction may be determined to be high. Thus, in this embodiment, when the width of the confidence interval (the difference between the upper confidence limit and the lower confidence limit) of all predicted values ​​is equal to or less than twice the tolerance, it may be determined that the uncertainty of the predicted value is tolerable. That is, as shown in FIG. 29, in this embodiment, the tolerance is 5 as set in the anomaly determination definition master 106b, and therefore the tolerance of the uncertainty of the predicted value is evaluated as being tolerable for deviation of 5 in both the positive and negative directions from the predicted value.Here, in this embodiment, if the width of the confidence interval is less than or equal to twice the tolerance, the uncertainty of the predicted value is determined to be acceptable, and if the uncertainty of all predicted values ​​is acceptable, the constructed prediction model may be used to determine anomalies, so subsequent processing continues. If there is even one predicted value whose uncertainty is determined to be unacceptable, subsequent processing is skipped and processing moves to the data of the next prediction unit, and the "Acceptable?" column is set to "Circle" if it is acceptable (in Figure 29, the tolerance = 5, so the tolerance x 2 = 10 or less), or "X" if it is not acceptable.

[0084] Then, as shown in Figures 30 to 32, in this embodiment, the confidence intervals of all predicted values ​​are evaluated to see if they are smaller than twice the tolerance range, and only the first two predicted values ​​are enlarged and displayed. For all predicted values, twice the length of the tolerance range arrow is compared with the length of the confidence interval width arrow. If the width of the confidence interval is smaller than twice the tolerance range for all predicted values, the uncertainty of the prediction results returned by the predictive model is deemed to be acceptable.

[0085] As shown in FIG. 32, in this embodiment, when the reliability of the parameter setting of the abnormality determination definition master 106b is specified as 0.68, which is smaller than 0.9, the uncertainty of the predicted value is more likely to be tolerated. In this manner, in this embodiment, as the reliability decreases, the uncertainty of the predicted value is more likely to be tolerated. That is, in this embodiment, as the reliability decreases, the confidence interval obtained from the upper confidence limit and the lower confidence limit returned from the time series prediction component becomes narrower, and the width of the confidence interval tends to become narrower than twice the value of the tolerance value, and the uncertainty of the predicted value is more likely to be tolerated. In addition, in this embodiment, if the reliability is not set, the process of "evaluating the tolerance of the uncertainty of the predicted value" is skipped, and the process proceeds to the abnormality determination process without evaluating the uncertainty. This is effective in cases where the purpose is to obtain a baseline of the future trend of the time series data without worrying about the uncertainty of the prediction, and the user's adjustment of the reliability is a parameter that adjusts the degree to which the uncertainty of the predicted value is tolerated to perform the abnormality determination.

[0086] Then, in this embodiment, the time series data of the prediction period created in the "creation of time series data for the prediction period" process is combined with the prediction result, as shown in Fig. 33. Here, as shown in Fig. 33, in this embodiment, the non-legal hours meter of the time series data of the prediction period contains NULL, so it is ignored.

[0087] Next, as shown in Fig. 34, in this embodiment, the purpose is to detect an anomaly in data where the predicted value may exceed a reference value. Here, as shown in Fig. 34, in this embodiment, if the predicted value of the non-legal hours meter exceeds the legally stipulated 45 hours, an anomaly for an employee is detected.

[0088] Specifically, in this embodiment, as shown in Fig. 34, a reference value is obtained from a setting item registered in the master, a prediction result is obtained, and since the reference value is 45, a prediction value of 45 or more is determined to be abnormal, and if abnormal, TRUE is set in the abnormality determination result column, and if not abnormal, FALSE is set in the abnormality determination result column. That is, in this embodiment, if a prediction value exceeds the reference value of 45, it is displayed as being capable of being determined to be abnormal, as shown in Fig. 35.

[0089] Then, as shown in FIG. 36, in this embodiment, the "preprocessing" to "abnormality determination processing" are performed for each group of all prediction units, and an abnormality determination result is obtained.

[0090] Next, as shown in Fig. 37, in this embodiment, the abnormality judgment result for each prediction unit has been obtained by the processing up to this point, and the purpose is to update the table with information required for displaying the judgment result. That is, as shown in Fig. 37, in this embodiment, by calculating and updating "reference value difference = predicted value - reference value", it becomes easier to obtain the value of how much the reference value is exceeded on a certain working day.

[0091] As shown in Figs. 38 and 39, the present embodiment aims to allow the user to effectively use the anomaly determination result by devising a display of the determination result. That is, the present embodiment aims to allow the user to visualize the information required by the user and take appropriate measures by displaying the actual value of the predicted time series data, the future predicted value, the confidence interval of the predicted value, and the reference value. As a result, in the present embodiment, when the predicted value exceeds the reference value, it is displayed so that the user can intuitively understand that it is abnormal, and appropriate measures can be taken. By displaying not only the predicted value but also the confidence interval of the predicted value, it is possible to determine whether or not there is a possibility of exceeding the reference value while grasping the reliability of the anomaly determination. In this embodiment, when abnormal data is found that exceeds a reference value based on the reliability and the reliability interval, (1) if the reliability is high (90%) and the reliability interval is wide, when viewed at a strict level (ruler) (when strict reliability is required), the fluctuation range of the predicted value is large, and although it is unclear whether the level is too strict or there is high uncertainty, care must be taken to avoid exceeding the reference value; (2) if the reliability is high (90%) and the reliability interval is narrow, when viewed at a strict level (ruler) (when strict reliability is required), the fluctuation range of the predicted value is small, and the predicted result appears to be reliable, so immediate action must be taken to avoid exceeding the reference value. (3) When the reliability is low (68%) and the reliability interval is wide, the predicted value fluctuates widely even at a loose level (ruler) (requiring a loose reliability), so the predicted value cannot be trusted very much and it is better to observe the progress of the time series data; (4) when the reliability is low (68%) and the reliability interval is narrow, the predicted value fluctuates narrowly when viewed at a loose level (ruler) (requiring a loose reliability), so although the predicted value fluctuates narrowly, the reliability of the prediction is low and therefore the accuracy of the prediction is not very high, so it is possible to take action such as paying attention to exceeding the standard value.

[0092] As shown in Figs. 38 and 39, in this embodiment, the actual value of the time series data to be predicted, the future predicted value, the confidence interval of the predicted value (the reliability of the confidence interval can be specified by the user), and the reference value are displayed as the abnormality determination result. In this embodiment, if the predicted value exceeds the reference value by the abnormality determination process, it is determined to be abnormal. However, if the confidence interval has a width of ± or more than the tolerance range from the predicted value, the reliability of the prediction result is deemed low, and the abnormality determination process is not performed, so the reliability of the predicted value and the abnormality determination result is guaranteed. In this embodiment, the actual value, the future predicted value, the confidence interval of the predicted value, and the reference value are displayed as the abnormality determination result, so that the user can compare it with the actual data through the graph display of the abnormality determination result, and by checking the confidence interval specified by the user and the difference between the trend of the predicted value and the reference value, it is possible to grasp whether there is a possibility of exceeding the reference value, and as a result, it is possible to take appropriate countermeasures.

[0093] [Time series prediction processing] Moreover, an example of the time series prediction process in this embodiment will be described with reference to Fig. 40 to Fig. 42. Fig. 40 is a diagram showing an example of information to be passed to a time series prediction component in this embodiment. Fig. 41 is a diagram showing an example of a decomposition result in this embodiment. Fig. 42 is a diagram showing an example of a prediction result in this embodiment.

[0094] As shown in Fig. 40, in this embodiment, the actual results, number of prediction points, window size, and reliability of the time series data to be predicted in the future are passed to the time series prediction component as inputs, and future data up to the number of prediction points of the actual results of the time series data to be predicted is predicted, and when making the prediction, the predicted value of the future data and a confidence interval according to the reliability received as input are also predicted. Note that the reliability does not have to be set, and if it is not set, the information returned by the time series prediction component does not include upper and lower confidence limits, and if it is set, the upper and lower limits of the confidence interval of the inner reliability of the predicted value are returned.

[0095] As shown in FIG. 41, in this embodiment, a lag matrix is ​​created, which is a matrix in which data shifted by the number of lags is arranged for the time series data, and singular value decomposition is performed on the lag matrix. The result of the singular value decomposition is then decomposed into each component, and the trend component, seasonal component, and residual component of the original time series data are calculated, and the trend component and seasonal component of the lag matrix are restored, so that the original time series data is decomposed and displayed as shown in a graph. Here, the residual component is calculated by subtracting the trend component and seasonal component from the original time series data. That is, as shown in FIG. 41, in this embodiment, the decomposition result when the window size is 30 is displayed, and the original time series data has a tendency to have increasingly higher peaks, but the trend component shows a tendency to monotonically increase, so this feature is well expressed, and it can also be seen that the periodicity of the seasonal component is captured. For comparison, as shown in Fig. 41, in this embodiment, the decomposition results for a window size of 10 are displayed, and it can be seen that the setting value of the window size is important in constructing a time series prediction model, since the decomposition is better when the window size is 30 than when the window size is 10. Note that, as shown in Fig. 41, in this embodiment, when the window size is 10, the trend component is not extracted as well as when the window size is 30, the trend of the trend component shows a periodic transition like the seasonal component, and the seasonal component transitions similarly to the residual component, and the decomposition of the seasonal component and the residual component is not performed well.

[0096] Then, as shown in FIG. 42, in this embodiment, a prediction model is constructed and learned for each of the trend component and the seasonal component, and finally, a prediction result of the original time series data is returned from the time series prediction component. Thus, in this embodiment, the window size has a large effect on the tendency of the trend component and the seasonal component, and if the window size is too small, the trend component reacts sensitively to small changes, resulting in overfitting and unstable model fitness of future prediction, and conversely, if the window size is too large, the trend component only captures gentle trends, and the seasonal component changes minutely close to the residual component. As a result, in this embodiment, the prediction is rough, with a large average confidence interval. For example, in this embodiment, if there is a one-month periodicity in the closing period, a better prediction may be possible by looking at shorter weekly or longer quarterly periodicity.

[0097] [Preprocessing] Moreover, an example of pre-processing in this embodiment will be described with reference to Fig. 43 to Fig. 45. Fig. 43 to Fig. 45 are diagrams showing an example of pre-processing in this embodiment.

[0098] As shown in FIG. 43, in this embodiment, the time series items to be interpolated are of the date type, and the data for Saturdays, Sundays, and holidays are interpolated (1. Interpolating the dates so that they are at equal intervals (in units of one day), and 2. Interpolating the other columns other than the dates (including days of the week) with the immediately preceding or succeeding values), to perform missing data interpolation on the date-type time series data for the accumulated overtime hours of an employee.

[0099] 44, in this embodiment, as a rounding process for the measurement error of the time series data (rounding process of the measurement time to the nearest whole number), rounding (discarding) is performed on the time series data that has a slight delay in the execution time, for example, the measurement time of the CPU utilization rate is rounded down to the nearest whole number, and it is assumed that a slight delay occurs each time the measurement is performed at ".00 seconds".

[0100] Then, as shown in FIG. 45, in this embodiment, time series data is expanded (for example, data measured every second is expanded to data measured every 0.5 seconds) when the sample size needs to be increased or when values ​​need to be interpolated. Linear interpolation is performed on the target time series data, and using two points (x1, y1) and (x2, y2) before and after the linear interpolation, a line segment passing through the two points: y=y1+(y2-y1) / (x2-x1)×(x-x1) (1), is calculated. When (x1, y1)=(0, 44), (x2, y2)=(1, 70), equation (1) becomes y=57 at x=0.5, and the interpolated value is (x, y)=(0.5, 57). For example, in this embodiment, when the measurement time of the CPU utilization rate is changed from 1 second units to 0.5 second units, if the input x is the measurement time and the output y (numerical data to be interpolated) is the CPU utilization rate, when (x1, y1) = (16:55:00.00, 43.69), (x2, y2) = (16:55:01.00, 69.6), formula (1) becomes x = 16:55:00.50 and y = 56.65. Here, as shown in Fig. 45, in this embodiment, the left table displays the transition of the CPU utilization rate for every 1 second of measurement time, and the right table displays the transition of the CPU utilization rate after the measurement time is interpolated every 0.5 seconds.

[0101] [Kernel decision processing] An example of the kernel determination process in this embodiment will be described with reference to Fig. 46 to Fig. 51. Fig. 46 to Fig. 48 are diagrams showing an example of a prior distribution in this embodiment. Fig. 49 to Fig. 51 are diagrams showing an example of a prediction function in this embodiment.

[0102] In this embodiment, in Bayesian optimization, the nature of the objective function to be targeted may be unknown, and it is necessary to set a kernel function that can increase the flexibility of the function and obtain an accurate optimal solution based on this. In this embodiment, the type of kernel function includes an exponential kernel and a periodic kernel, but a Gaussian kernel is adopted, and the value of its length parameter (hereinafter, coefficient) is 5. Here, the kernel function is a function for calculating the similarity of input data of two points (x1, x2), and is a type of covariance function, which represents the correlation between input variables. Strictly speaking, the kernel function is expressed by applying each of x1 and x2 to a nonlinear function and expressing the inner product of the functions. Also, as shown in FIG. 46 to FIG. 48, in this embodiment, the nature of the Gaussian kernel is that the Gaussian kernel k is expressed as a two-variable function, and the value attenuates exponentially like a normal distribution according to the distance between the two points x1 and x2. Here, x1 and x2 represent candidate points, and the smoothness of the Gaussian kernel can be adjusted by changing the value of the coefficient ρ. As shown in Figs. 46 to 48, in this embodiment, samples of the prior distribution when the value of ρ is changed are shown, and the prior distribution gives a tentative predicted value of the objective function, the shape of the objective function is assumed in advance, and ρ is selected from the prior distribution sample. In addition, in this embodiment, the larger ρ is, the larger the value of k is, and the larger the covariance between the two points x1 and x2 is, and since "the larger the covariance is, the closer the output of the two points is to a value," the function becomes smooth overall. On the other hand, in this embodiment, the smaller ρ is, the more likely the value of k is to approach 0, and the more likely the covariance between the two points x1 and x2 is to approach 0, and as a result, each point becomes independent, resulting in a function with low smoothness. Furthermore, in this embodiment, the Gaussian kernel is adopted because it allows for flexible expression of functions (because the Gaussian kernel is created from an infinite-dimensional feature vector), and because the kernel function is defined as the dot product of feature vectors and can be expressed as a scalar (real number), there is no problem even if the dimension of the feature vector is infinite (the dot product of 1×n vectors is 1×n, n×1, which is a 1×1 scalar, so the size of n is irrelevant).Here, the feature vector is a vector having basis functions as elements. In addition, the reason why other kernels were not selected in this embodiment is that the linear kernel cannot flexibly express functions, the exponential kernel lacks smoothness and cannot flexibly express functions, and the periodic kernel can flexibly express functions but cannot necessarily assume periodicity in the objective function to be estimated.

[0103] In this embodiment, the coefficient ρ of the Gaussian kernel is determined as follows, in order to obtain an optimal solution within a limited calculation time, if "the number of candidate points to be searched÷5≦the number of candidate points that can be searched within the time", the coefficient ρ=5, and if "the number of candidate points to be searched÷5>the number of candidate points that can be searched within the time", the coefficient ρ=[the number of candidate points to be searched]÷[the number of candidate points that can be searched within the time]". Here, as shown in Fig. 49 to Fig. 51, in this embodiment, graphs of the prediction function and standard deviation function predicted when ρ=2, 5, and 10 are shown. That is, in this embodiment, as shown in Fig. 49, when ρ=2, search points are searched intensively around window sizes 30 to 40, and are not searched broadly, and as shown in Fig. 50, when ρ=5, search is focused around window sizes 40 to 50 where the optimal solution is likely to be found, and window sizes 1 to 30 are also searched moderately, and as shown in Fig. 51, when ρ=10, search is performed broadly, but since the uncertainty (standard deviation) is displayed in an understated manner, it takes time to find a truly optimal solution. From the above, in this embodiment, ρ=5 is basically set, and only when "the number of candidate points to be searched ÷ 5>the number of candidate points that can be searched in time", a large ρ is set so that a wide search is performed and a value as close to the truly optimal value as possible can be selected.

[0104] [4. Contribution to the United Nations-led Sustainable Development Goals (SDGs)] This embodiment can contribute to improving business efficiency and promoting appropriate management decisions by companies, thereby contributing to goals 8 and 9 of the SDGs.

[0105] Furthermore, this embodiment can contribute to reducing waste and promoting paperless and electronic systems, thereby contributing to the achievement of goals 12, 13, and 15 of the SDGs.

[0106] Furthermore, this embodiment can contribute to strengthening control and governance, making it possible to contribute to Goal 16 of the SDGs.

[0107] 5. Other embodiments The present invention may be embodied in various different embodiments other than those described above within the scope of the technical concept set forth in the claims.

[0108] For example, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods.

[0109] In addition, the processing procedures, control procedures, specific names, registered data for each process, information including parameters such as search conditions, screen examples, and database configurations shown in this specification and drawings may be changed as desired unless otherwise specified.

[0110] Moreover, with regard to the time series prediction management device 100, each of the components shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure.

[0111] For example, the processing functions of the time series prediction management device 100, particularly the processing functions performed by the control unit 102, may be realized in whole or in part by a CPU and a program interpreted and executed by the CPU, or may be realized as hardware using wired logic. The program is recorded in a non-transient computer-readable recording medium that includes programmed instructions for causing an information processing device to execute the processes described in this embodiment, and is mechanically read by the time series prediction management device 100 as necessary. That is, a computer program for giving instructions to the CPU in cooperation with the OS and performing various processes is recorded in a storage unit such as a ROM or HDD (Hard Disk Drive). This computer program is executed by being loaded into a RAM, and cooperates with the CPU to form a control unit.

[0112] In addition, this computer program may be stored in an application program server connected to the time series prediction management device 100 via an arbitrary network, and it is also possible to download all or a part of it as necessary.

[0113] In addition, the program for executing the process described in this embodiment may be stored in a non-transient computer-readable recording medium, or may be configured as a program product. Here, the "recording medium" includes any "portable physical medium" such as a memory card, a Universal Serial Bus (USB) memory, a Secure Digital (SD) card, a flexible disk, a magneto-optical disk, a ROM, an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable and Programmable Read Only Memory (EEPROM (registered trademark)), a Compact Disk Read Only Memory (CD-ROM), a Magneto-Optical disk (MO), a Digital Versatile Disk (DVD), and a Blu-ray (registered trademark) Disc.

[0114] Moreover, a "program" is a data processing method written in any language or description method, and may be in any form, such as source code or binary code. Note that a "program" is not necessarily limited to a single configuration, but also includes a configuration that is distributed as a plurality of modules or libraries, and a program that achieves its function by cooperating with a separate program, such as an OS. Note that the specific configuration and reading procedure for reading a recording medium in each device shown in this embodiment, as well as the installation procedure after reading, may use well-known configurations and procedures.

[0115] The various databases etc. stored in the memory unit 106 are storage means such as memory devices such as RAM and ROM, fixed disk devices such as hard disks, flexible disks, and optical disks, and store various programs, tables, databases, and web page files etc. used for various processes and providing websites.

[0116] The time series prediction management device 100 may be configured as an information processing device such as a known personal computer or a workstation, or may be configured as the information processing device to which any peripheral device is connected. The time series prediction management device 100 may be realized by installing software (including programs, data, etc.) that causes the device to realize the processing described in this embodiment.

[0117] Furthermore, the specific form of distribution and integration of the devices is not limited to that shown in the drawings, and all or part of them can be functionally or physically distributed and integrated in any unit according to various additions or functional loads. In other words, the above-mentioned embodiments can be implemented in any combination, or the embodiments can be implemented selectively. [Industrial Applicability]

[0118] The present invention is useful in various industries that use time-series data such as attendance management. [Explanation of symbols]

[0119] 100 Time series prediction management device 102 Control section 102a Model Creation Section 102b Time series prediction section 102c Hyperparameter determination unit 102d Model Conformity Assessment Division 102e Uncertainty Tolerance Assessment Section 102f Anomaly detection section 102g Result display section 104 Communication interface section 106 Storage section 106a Time Series Database 106b Abnormality Judgment Definition Master 108 Input / Output Interface Section 112 Input Devices 114 Output Device 200 Servers 300 Network

Claims

1. A time series prediction management device including a storage unit and a control unit, The storage unit is a time series storage means for storing time series data in which a performance value depending on time is set; an anomaly determination definition master in which a reference value of the time series data and a reliability and a tolerance range of a predicted value of the time series data are set; Equipped with The control unit is A model creation means for creating a time series prediction model, which is a machine learning model, using a part of the time series data as training data; a time series prediction means for obtaining the predicted value of the time series data by using the time series prediction model and obtaining a confidence interval for the predicted value based on the abnormality determination definition master; a result display means for displaying the reference value, the predicted value, and the confidence interval of the time-series data in time series; A time series prediction management device comprising:

2. The model creation means includes: The time series prediction management device according to claim 1, characterized in that the time series data is divided into the training data, validation data, and test data, and the time series prediction model, which is the machine learning model, is created using the training data.

3. The control unit is a hyperparameter determination means for determining a hyperparameter having high adaptability for the time series prediction model by comparing the validation data with the predicted value of the validation data; The time series prediction management device according to claim 2, further comprising:

4. The control unit is a model suitability evaluation means for obtaining a model suitability evaluation indicating that the suitability of the time series prediction model is sufficient when a ratio of the predicted values ​​of the test data included in the confidence interval is equal to or greater than a predetermined ratio; The time series prediction management device according to claim 2, further comprising:

5. The control unit is an uncertainty tolerance assessment means for obtaining an uncertainty tolerance assessment indicating that the uncertainty of the predicted value is tolerable if the width of the confidence interval is narrower than twice the tolerance width; 5. The time series prediction management device according to claim 1, further comprising:

6. A time series prediction management method to be executed by a time series prediction management device having a storage unit and a control unit, The storage unit is a time series storage means for storing time series data in which a performance value depending on time is set; an anomaly determination definition master in which a reference value of the time series data and a reliability and a tolerance range of a predicted value of the time series data are set; Equipped with Executed in the control unit: a model creation step of creating a time series prediction model, which is a machine learning model, using a part of the time series data as training data; a time series prediction step of acquiring the predicted value of the time series data using the time series prediction model and acquiring a confidence interval for the predicted value based on the anomaly determination definition master; a result display step of displaying the reference value, the predicted value, and the confidence interval of the time-series data in a time-series manner; A time series prediction management method comprising:

7. A time series prediction management program to be executed by a time series prediction management device having a storage unit and a control unit, The storage unit is a time series storage means for storing time series data in which a performance value depending on time is set; an anomaly determination definition master in which a reference value of the time series data and a reliability and a tolerance range of a predicted value of the time series data are set; Equipped with In the control unit, a model creation step of creating a time series prediction model, which is a machine learning model, using a part of the time series data as training data; a time series prediction step of acquiring the predicted value of the time series data using the time series prediction model and acquiring a confidence interval for the predicted value based on the anomaly determination definition master; a result display step of displaying the reference value, the predicted value, and the confidence interval of the time-series data in a time-series manner; A time series forecast management program for executing the above.

Images