Information processing apparatus, information processing method, and storage medium

Abstract

An information processing apparatus, an information processing method, and a storage medium improve the accuracy of estimation using a model. The information processing apparatus includes a processing unit. The processing unit inputs input feature amounts obtained at a first input period to a plurality of models learned using values of hyperparameters that differ from each other in a plurality of groups including at least one of the values of the hyperparameters, and calculates first estimated values of object feature amounts in a first object period. The processing unit selects one or more groups from the plurality of groups using evaluation values based on differences between the first estimated values and actual values of the object feature amounts obtained at the first object period. The processing unit inputs input feature amounts obtained at a second input period after the first input period to the plurality of models, and calculates second estimated values of the object feature amounts in a second object period. The processing unit outputs one or more second estimated values output from one or more models corresponding to the selected one or more groups.

Description

Technical Field

[0001] The embodiments of the present invention relate to information processing apparatus, information processing method, and storage medium. Background Technology

[0002] The following techniques are known: data obtained during a certain period (time series data) is input into a model (machine learning model, deep learning model) obtained through machine learning to infer data for subsequent periods. Furthermore, to reduce hardware resource usage and achieve efficient inference, a technique has been proposed to dynamically select the appropriate model from multiple models according to the inference conditions. In inferences using models, there is sometimes a need to improve inference accuracy. Summary of the Invention

[0003] The purpose of this invention is to provide an information processing apparatus, an information processing method, and a storage medium that can improve the accuracy of predictions using models.

[0004] The information processing apparatus of this embodiment includes a processing unit. For multiple models learned using hyperparameter values ​​from multiple groups whose hyperparameter values ​​differ from each other, the processing unit inputs input feature quantities obtained in a first input time period into each model and calculates a first predicted value for object feature quantities in a first object time period. The processing unit selects one or more groups from the multiple groups using an evaluation value based on the difference between the first predicted value and the actual value of the object feature quantity obtained in the first object time period. For each of the multiple models, the processing unit inputs input feature quantities obtained in a second input time period after the first input time period into each model and calculates a second predicted value for the object feature quantities in a second object time period. The processing unit outputs one or more second predicted values ​​from one or more models corresponding to the selected one or more groups. Attached Figure Description

[0005] Figure 1 This is a block diagram of the information processing apparatus according to the implementation method.

[0006] Figure 2 This is a graph illustrating an example of learning using time series data.

[0007] Figure 3 This is a graph showing an example of forecasting using time series data.

[0008] Figure 4 This is a flowchart of the learning process in the implementation method.

[0009] Figure 5 This is a flowchart of the prediction process in the implementation method.

[0010] Figure 6 This is a diagram illustrating an example of the data structure for predicting values.

[0011] Figure 7 This is a hardware structure diagram of the information processing device in the implementation method.

[0012] Figure Labels

[0013] 100: Information processing device; 101: Acquisition unit; 102: Output control unit; 110: Storage unit; 120: Learning control unit; 121: Creation unit; 122: Learning unit; 130: Prediction control unit; 131: Creation unit; 132: Prediction unit; 133: Selection unit. Detailed Implementation

[0014] The preferred embodiments of the information processing apparatus of the present invention will be described in detail below with reference to the accompanying drawings.

[0015] In the following implementation, we will primarily illustrate an example of inferring data for periods following a given time period by inputting time series data obtained at a certain time period into a model. Furthermore, the term "prediction" will sometimes be used instead of "inference" below. "Prediction" is not limited to finding future data. For example, the period following the time period into which the data to be input into the model is obtained is not limited to a period in the future of the processing time point, but could also be a period in the past of the processing time point.

[0016] The data used for prediction and the time series data used for prediction can be any data. For example, the data used for prediction can be the water inflow into a reservoir at a specific location (reservoir inflow) and meteorological data such as wind speed at that location. The following primarily illustrates an example where the data used for prediction is reservoir inflow. In this case, the time series data used for prediction may include, for example, actual rainfall values ​​at one or more locations (such as the center of a 5km grid) within the reservoir's inflow area.

[0017] In model learning, randomness exists in aspects such as the initialization of learning parameters and the shuffling of learning data during mini-batch learning. This randomness is determined by, for example, a random number seed. To maintain model reproducibility, a fixed value is typically used as the random number seed. The random number seed can be interpreted as an example of hyperparameters pre-set before learning. Hyperparameters are not limited to random number seeds; they can also be other parameters such as the learning rate. Because changes in the values ​​of other hyperparameters also affect the prediction results, fixed values ​​are usually used for model learning. In this way, in the learning of a predictive model, a group of pre-set values ​​for one or more hyperparameters is used.

[0018] When using a model (predictive model) learned with the hyperparameters set as described above to predict time series data (time series forecasting), the accuracy of the predictive model can vary significantly depending on the nature of the learning data and the set of hyperparameter values. That is, the accuracy of the learned predictive model can sometimes change when different sets of hyperparameter values ​​are used for learning.

[0019] One commonly cited reason is that when using different sets of hyperparameter values, the predictive models converge to different minima in the landscape of the loss function during the learning process. As a result, multiple predictive models corresponding to different sets of hyperparameter values ​​become models that primarily learn different time-series features within the learning data. That is, for each of the multiple models, the periods they are good at predicting and the periods they are not good at predicting differ within the evaluation period. Consequently, the predictive accuracy of the multiple models varies. Furthermore, the evaluation period refers to the time period during which predictions are made using the learned predictive models. The evaluation period can be of any length, such as one year.

[0020] When using multiple models with varying prediction accuracies, a common approach to improve accuracy is to integrate the predictions obtained from these models (ensemble, averaging, etc.). However, this method may result in an overall prediction accuracy that tends towards a moderate level.

[0021] In order to improve the accuracy of predictions using models, the information processing device in the implementation method has, for example, the following functions.

[0022] • Use multiple prediction models corresponding to multiple groups whose values ​​are different from each other for at least some of the hyperparameters.

[0023] • At a certain point in time within the evaluation period, when performing a prediction for the object period OB following the object period OA included in the evaluation period, a group of hyperparameter values ​​of the prediction model used in the object period OB is selected based on the prediction values ​​of multiple prediction models in the object period OA and the actual values ​​obtained for the object period OA.

[0024] • Output the predicted values ​​of the prediction model learned using the values ​​of the hyperparameters contained in the selected groups as the predicted values ​​in the object time period OB.

[0025] Repeat the above process until the end of the evaluation period.

[0026] Furthermore, the object period refers to the time period used to calculate the predicted value of the data (such as reservoir inflow) that is the object of prediction. The object period is a time period included in the evaluation period, which is equivalent to a time period set from the beginning to the end of the evaluation period. For example, if the evaluation period is 1 year and the object period is 3 hours long, the time period of 3 hours from the beginning of the evaluation period is set as the object period in sequence, and the prediction of the predicted value is repeated.

[0027] Using the above functions, problems such as poor accuracy due to differences in the values ​​of the hyperparameters used for learning and the normalization of prediction accuracy due to ensemble methods are avoided. It can dynamically output prediction values ​​that correspond to the situation, thereby obtaining high prediction accuracy.

[0028] Furthermore, this embodiment can be interpreted as follows: considering the predicted values ​​for a certain target time period, the best predicted value among multiple predicted values ​​in the next target time period is selected and output. Therefore, for example, when the time when the properties of the data being predicted change is later than the time that causes the target time periods to be staggered (3 hours in the example above), the prediction accuracy can be further improved. In other words, it is preferable to set the length of the target time period considering the time when the properties of the data being predicted change.

[0029] Figure 1 This is a block diagram illustrating an example of the structure of the information processing apparatus 100 according to this embodiment. Figure 1 As shown, the information processing device 100 includes an acquisition unit 101, an output control unit 102, a storage unit 110, a learning control unit 120, and a prediction control unit 130.

[0030] The acquisition unit 101 acquires various types of information used in the information processing apparatus 100. For example, the acquisition unit 101 acquires data (time series data, etc.) used for learning the prediction model and making predictions by the prediction model.

[0031] The method by which the acquisition unit 101 acquires information can be any method, such as receiving information from an external device via a network or reading information from a storage medium.

[0032] The output control unit 102 controls the output of various types of information used in the information processing device 100. For example, the output control unit 102 outputs the final predicted value for each target time period obtained by the prediction control unit 130. The information output method can be any method, such as a method that can be applied to a display device for display or a method that sends information to an external device via a network.

[0033] The storage unit 110 stores various types of information used in the information processing device. For example, the storage unit 110 stores hyperparameters 111, prediction models 112, auxiliary features 113, prediction features 114, auxiliary features 115, and prediction features 116.

[0034] Hyperparameter 111 is the parameter used in the learning of prediction model 112 and in the predictions made by prediction model 112. Hyperparameter 111 can be any parameter, including, for example, the hyperparameters required to create the dataset for learning or prediction and the hyperparameters used for learning prediction model 112.

[0035] Hyperparameters required to create the dataset include, for example, the length of the input data sequence (hereinafter referred to as the input sequence length) and the length of the object time period. Hyperparameters used for learning include, for example, the random number seed, learning rate, batch size, and the hidden layer size of the prediction model 112.

[0036] In this embodiment, at least a portion of the hyperparameters described above are used as switching object hyperparameters (hereinafter referred to as switching object parameters). Switching object parameters represent hyperparameters with different values ​​set for obtaining multiple prediction models 112 through learning. A group of values ​​for one or more switching object parameters corresponds to a group of values ​​for one or more hyperparameters described above.

[0037] The following example primarily illustrates the use of only a random number seed as a switching parameter. Switching parameters can be combinations of multiple hyperparameters of different types, such as (random number seed, learning rate, input sequence length).

[0038] The predictive model learned by changing the values ​​of the switching object parameters can be a single, general predictive model or multiple, non-general predictive models. A general predictive model refers to, for example, models with the same architecture. Multiple, non-general predictive models refer to, for example, multiple models with different architectures.

[0039] The term "architecture" refers to the structure of a model, such as its layer structure, number of layers, connection patterns, and activation functions. The size of hidden layers can be interpreted as part of the architecture. For hyperparameters that can be interpreted as part of the architecture, models with the same architecture except for those hyperparameters can be treated as a single, generic prediction model.

[0040] Prediction model 112 can be a model with any architecture, such as the following architecture.

[0041] • Models based on LSTM (Long Short Term Memory) (LSTM model)

[0042] • Recurrent neural network model

[0043] State-space model

[0044] • Converter

[0045] Models based on decision tree algorithm

[0046] This section will illustrate an example of using predictive models with the same architecture. Assume the random number seed (the switching parameter) is set to one of five values: 1, 3, 5, 7, or 9. For example, in the initial learning process, 1 is set as the random number seed value. In this example, by learning using five different values ​​as the switching parameter values ​​for predictive models with the same architecture, five predictive models are obtained.

[0047] This section will illustrate an example using multiple prediction models with different architectures. The switching parameters for each prediction model can be either the same or different. In the latter case, for example, the switching parameter set for prediction model A1 could be set to (random number seed, learning rate, input sequence length), while the switching parameter set for prediction model A2 could be set to (batch size). In this example, a number of prediction models is obtained that is equivalent to the sum of the number of switching parameter sets for prediction model A1 and prediction model A2.

[0048] Hyperparameter 111 can be associated with information indicating whether it is a switching target parameter. Based on this, it is possible to determine which of the hyperparameters 111 is a switching target parameter. Storage unit 110 stores the number of sets of values ​​for the hyperparameters 111 that are switching target parameters. In the example using 5 random number seeds, storage unit 101 stores the values ​​1, 3, 5, 7, and 9.

[0049] Prediction model 112 is equivalent to a prediction model obtained through learning. Storage unit 110 stores the prediction model 112 in the number of sets of values ​​for the switching object parameters. In the example using 5 random number seeds, storage unit 110 stores 5 prediction models 112 corresponding to the 5 values ​​1, 3, 5, 7, and 9.

[0050] Auxiliary feature 113 represents the auxiliary feature (time series data) used during learning. Predictive feature 114 represents the predictive feature (time series data) used during learning. Auxiliary feature 115 represents the auxiliary feature (time series data) used during prediction. Predictive feature 116 represents the predictive feature (time series data) used during prediction.

[0051] When there is no need to distinguish between learning time and prediction time, auxiliary feature quantity 113 and auxiliary feature quantity 115 are sometimes simply referred to as auxiliary feature quantity, and prediction feature quantity 114 and prediction feature quantity 116 are simply referred to as prediction feature quantity.

[0052] Predictive features refer to the features used as the object of prediction (object features). Auxiliary features refer to features other than predictive features used for learning or prediction. Furthermore, the features input to prediction model 112 will be referred to as input features. For example, input features may include both predictive features and auxiliary features. Input features may also include either predictive features or auxiliary features.

[0053] For example, if the example is predicting reservoir inflow, the input features and prediction features would be as follows.

[0054] • Input features: Reservoir inflow (actual value), and actual rainfall values ​​at more than one location (such as the center of a 5km grid) within the reservoir's inflow area (equivalent to auxiliary features).

[0055] • Predicted feature: Reservoir inflow (the inflow into the reservoir during the period following the period in which the actual value of the reservoir inflow included in the input feature is obtained).

[0056] As an input feature, it can replace the actual value or be in addition to the actual value, including the predicted value obtained in advance for the target time period.

[0057] Furthermore, the storage unit 110 can be composed of all commonly used storage media such as flash memory, memory cards, RAM (Random Access Memory), HDD (Hard Disk Drive), and optical discs.

[0058] The data (hyperparameter 111, prediction model 112, auxiliary feature quantity 113, prediction feature quantity 114, auxiliary feature quantity 115, and prediction feature quantity 116) stored in the storage unit 110 can be stored in some or all of physically different storage media or in different storage areas of the same physical storage medium.

[0059] The data stored in the storage unit 110 is not limited to the above-mentioned content, and other data can be stored in the storage unit 110. For example, the storage unit 110 can store the processing results of various units such as the learning control unit 120 and the prediction control unit 130 (data created by the learning control unit 120 when learning, prediction results obtained by the prediction control unit 130, etc.).

[0060] The learning control unit 120 controls the learning of the prediction model 112. The learning control unit 120 includes a creation unit 121 and a learning unit 122.

[0061] The creation unit 121 creates learning time-series data for learning the prediction model 112. For example, the creation unit 121 reads auxiliary feature quantity 113 and prediction feature quantity 114 from the storage unit 110 and creates learning time-series data including the read auxiliary feature quantity 113 and prediction feature quantity 114. The learning time-series data is, for example, time-series data representing the input feature quantities input to the prediction model 112 during learning.

[0062] Figure 2 This diagram illustrates an example of learning time-series data created by the creation unit 121. The learning time-series data includes date and time, actual reservoir inflow values, and actual rainfall values ​​at locations P1 to P10. The date and time are set as the time period (learning period) from which the time-series data is obtained. Figure 2 The learning period is from 00:30:00 on January 1, 2019 to 00:00:00 on January 1, 2023. The length of the learning period is not limited to this.

[0063] Figure 2 The example shown has 10 locations where the actual rainfall values ​​(auxiliary feature values) are obtained, but the number of locations is not limited to 10. Figure 2 The example shown illustrates obtaining individual input features every 30 minutes, but the time interval (frame) for obtaining input features is not limited to 30 minutes. The representation of date and time is not limited to... Figure 2 The method shown can be any method.

[0064] The learning unit 122 uses the created learning time-series data to learn the prediction model 112. For example, the learning unit 122 reads the switching object parameters (hyperparameters 111) from the storage unit 110 and sets the read switching object parameters. Additionally, the learning unit 122 receives the learning time-series data from the creation unit 121. The learning unit 122 uses at least a portion of the learning time-series data as input data (input features) to the prediction model 112, creating a dataset (input dataset) that includes the input data and the correct answer data. For example, the sequence length of the input data is set according to the switching object parameters. The learning unit 122 uses the input dataset to enable the prediction model 112 to learn.

[0065] In this embodiment, the learning unit 122 sets the switching object parameters according to the number of sets of values ​​for the switching object parameters and learns the prediction model 112 using the input dataset. This constructs a prediction model 112 based on the number of sets of values ​​for the switching object parameters. The learning unit 122 stores the learned prediction model 112 in the storage unit 110.

[0066] The prediction control unit 130 controls the predictions made using the learned prediction model 112. The prediction control unit 130 includes a creation unit 131, a prediction unit 132, and a selection unit 133.

[0067] The creation unit 131 creates prediction time series data for prediction by the prediction model 112. For example, the creation unit 131 reads auxiliary feature quantities (e.g., the latest auxiliary feature quantity 115) and prediction feature quantities 116 (e.g., the latest prediction feature quantity 116) for prediction from the storage unit 110, and creates prediction time series data including the read auxiliary feature quantity 115 and prediction feature quantity 116. The prediction time series data is, for example, time series data representing the input feature quantities input to the prediction model 112 during prediction.

[0068] Figure 3 This is a diagram showing an example of time series data for prediction created by the creation unit 131. (Compared to time series data for learning...) Figure 2 Similarly, the forecast time series data includes date and time, actual reservoir inflow, and actual rainfall at locations P1 to P10. The date and time are set for the period in which the forecast time series data is obtained. Figure 3 The example illustrates how, assuming the evaluation period is one year from "2023-01-01 00:30:00 to 2024-01-01 00:00:00", the 30 days preceding the evaluation period (2022-12-02 00:30:00 to 2023-01-01 00:00:00) are used as the period for obtaining the forecast time series data. The length of the period for obtaining the forecast time series data is not limited to 30 days.

[0069] and Figure 2 Similarly, the number of locations from which the actual rainfall values ​​for the predicted time series data are obtained is not limited to 10, the time interval (frames) for obtaining the input features is not limited to 30 minutes, and the method of representing date and time is not limited to... Figure 3 The method shown.

[0070] The prediction unit 132 uses the created prediction time series data to make predictions using the prediction model 112. For example, for multiple prediction models 112, the prediction unit 132 inputs the input feature quantities obtained in the input time period IA (first input time period) into the prediction model 112 respectively, and calculates the predicted value EVA (first inferred value) of the predicted feature quantities in the target time period OA (first target time period) after the input time period IA, which is output from the prediction model 112. The prediction unit 132 stores the predicted value EVA of each prediction model 112 in, for example, the storage unit 110 and sends it to the output control unit 102.

[0071] The input time period can be any time period preceding the target time period. For example, input time period IA and input time period IB can be the time periods immediately preceding (most recent) the target time periods OA and OB, respectively. Furthermore, input time period IB represents the time period from which the input feature quantities used to calculate the predicted feature quantities in the target time period OB are obtained. A certain time period can be left between input time period IA (input time period IB) and target time period OA (target time period OB).

[0072] When predicting a feature that changes at regular intervals, for example, the input period IA can be a period corresponding to the target period OA, which is included in the period preceding the current period OA. Similarly, the input period IB can be a period corresponding to the target period OB, which is included in the period preceding the current period OB. For instance, when predicting a feature that changes over a yearly period, the input period IB can be a period (same month, same season, same week, etc.) that is included in the year preceding the current year, including the target period OB.

[0073] The selection unit 133 uses the predicted value obtained for a certain target time period to select a group of values ​​of the switching target parameter used in the prediction of the next target time period. For example, the selection unit 133 selects one or more groups from multiple groups based on an evaluation value based on the difference between the predicted value EVA obtained using the corresponding prediction model 112 and the actual value of the predicted feature quantity obtained in the target time period OA. Information indicating the selected group is stored, for example, in the storage unit 110.

[0074] Evaluation values ​​can be based on any metric. For example, evaluation values ​​can be represented by any one or more of the following values.

[0075] (I1) Mean Absolute Error (MAE) between predicted and actual values

[0076] (I2) Mean Absolute Percentage Error (MAPE) between predicted and actual values.

[0077] (I3) Mean Absolute Scaled Error (MASE) between predicted and actual values

[0078] (I4) Mean Squared Error (MSE) between predicted and actual values

[0079] (I5) Root Mean Square Error (RMSE) between predicted and actual values

[0080] (I6) Continuous Ranked Probability Score (CRPS) between predicted and actual values

[0081] (I7) Coefficient of determination R based on the difference between predicted and actual values 2

[0082] When using evaluation values ​​(I1) to (I6), the selection unit 133 selects, for example, the random number seed with the smallest evaluation value. When using the evaluation value (I7), the selection unit 133 selects, for example, the random number seed with the evaluation value closest to 1. The evaluation value can also be represented by an index (such as similarity) where a smaller difference results in a larger value. In this case, the selection unit 133 selects, for example, the switching object parameter with the largest evaluation value. When selecting multiple groups, the selection unit 133 can also select a certain number of groups, for example, in ascending order (or descending order) of evaluation values.

[0083] When multiple groups are selected, the selection unit 133 can calculate the weight (proportion) of each group and store the calculated weights in the storage unit 110, etc. The weights are used to calculate the final output predicted value (final predicted data) based on the multiple predicted values ​​corresponding to the selected multiple groups.

[0084] The weights can be calculated using any method. For example, the following method can be used: a portion of the learning period used to predict model 112 is set as the validation period, and the weights are determined based on the prediction results of the validation period.

[0085] The weights can be determined using the following method. Let's take the case where random number seeds 1, 3, and 5 are selected as an example. Assume that the errors (examples of evaluation values) between the predicted and actual values ​​corresponding to random number seeds 1, 3, and 5 are calculated as E1, E3, and E5, respectively. Selection unit 133 calculates the weights W1, W3, and W5 corresponding to random number seeds 1, 3, and 5 according to the following formula. W1, W3, and W5 are equivalent to the normalized values ​​of w1, w3, and w5.

[0086] w1=1 / E1, w3=1 / E3, w5=1 / E5

[0087] W1 = w1 / (w1 + w3 + w5)

[0088] W3 = w3 / (w1 + w3 + w5)

[0089] W5 = w5 / (w1 + w3 + w5)

[0090] The time period used to calculate the predicted and actual values ​​for the evaluation value is not limited to one object time period (e.g., the object time period immediately preceding the next object time period), but can also be multiple object time periods (e.g., multiple object time periods including the object time period immediately preceding the next object time period). For example, the evaluation value based on the difference between the predicted value EVA and the actual value of the predicted feature quantity obtained in the object time period OA can also be an evaluation value that takes into account the difference between the predicted and actual values ​​in the time period preceding the object time period OA.

[0091] The selected group is used by the prediction unit 132 to calculate the final prediction data based on the predicted values ​​for the next target time period. Hereinafter, the value of the switching target parameter corresponding to the selected group is sometimes referred to as the optimal value of the hyperparameter.

[0092] First, the prediction unit 132 sequentially advances the target time period within the evaluation period, thereby repeating the above process. For example, for multiple prediction models 112, the prediction unit 132 inputs the input feature quantity obtained in the input time period IB (the second input time period) after the input time period IA into the prediction model 112, and calculates the predicted value EVB (the second inferred value) of the predicted feature quantity of the target time period OB (the second target time period) after the input time period IB, which is output from the prediction model 112.

[0093] The prediction unit 132 uses the group (optimal value) selected by the selection unit 133 to determine the final output for the target time period OB. For example, the prediction unit 132 outputs more than one predicted value EVB from more than one prediction model 112 corresponding to more than one selected group as the final prediction result (final prediction data).

[0094] When multiple groups are selected by the selection unit 133, the prediction unit 132 can calculate multiple predicted values ​​EVB output from multiple prediction models 112 corresponding to the selected multiple groups, and output the value based on the multiple predicted values ​​EVB as the predicted value of the predicted feature quantity in the target time period OB.

[0095] The prediction unit 132 calculates and outputs statistical values ​​of multiple predicted values ​​EVB as the final prediction data. The statistical values ​​are, for example, a simple average (simple ensemble average), a weighted average (weighted ensemble average), and the median. The weights used for the weighted average (the weights assigned to each predicted value) can be values ​​calculated by the selection unit 133 and stored in the storage unit 110, etc.

[0096] At least some of the aforementioned units (acquisition unit 101, output control unit 102, learning control unit 120, and prediction control unit 130) can be implemented by one or more processing units. For example, each of the aforementioned units can be implemented using one or more processors. For instance, each of the aforementioned units can be implemented by having processors such as CPUs (Central Processing Units) and GPUs (Graphics Processing Units) execute programs, i.e., software. Each of the aforementioned units can also be implemented using processors such as application-specific integrated circuits (ASICs), i.e., hardware. Each of the aforementioned units can also be implemented using a combination of software and hardware. When using multiple processors, each processor can implement one or more of the aforementioned units.

[0097] The information processing device 100 can be physically comprised of a single device or multiple devices. For example, the information processing device 100 can be built in a cloud environment. Furthermore, the components within the information processing device 100 can be distributed across multiple devices. For instance, the information processing device 100 (information processing system) can be configured to include a device (e.g., a learning device) having functions required for learning the prediction model 112 (such as a learning control unit 120) and a device (e.g., a reasoning device) having functions required for reasoning using the prediction model 112 (such as a prediction control unit 130).

[0098] Next, the learning process performed by the information processing apparatus 100 in this embodiment will be described. The learning process is the process of enabling the prediction model 112 to learn. Figure 4 This is a flowchart illustrating an example of the learning process in this embodiment.

[0099] The creation unit 121 reads time-series data of auxiliary feature quantity 113 and prediction feature quantity 114 from the storage unit 110 (step S101). The creation unit 121 uses the read time-series data to create, for example... Figure 2 The structure shown is learned using time series data (step S102).

[0100] The learning unit 122 reads and sets the switching target parameter (hyperparameter 111) for the unprocessed group from the storage unit 110 (step S103). When using a group with 5 values ​​of the random number seed as the switching target parameter, the learning unit 122 reads and sets the value of the random number seed corresponding to the 1 unprocessed group.

[0101] The learning unit 122 creates an input dataset for learning the prediction model 112 based on the time series data, and uses the input dataset to enable the prediction model 112 to learn (step S104). The learning unit 122 stores the prediction model 112 obtained through learning in the storage unit 110 (step S105).

[0102] The learning unit 122 determines whether all groups of switch object parameter values ​​have been processed (step S106). If not, it returns to step S103 and repeats the processing for the next unprocessed group.

[0103] If all groups of switch object parameter values ​​have been processed (step S106: Yes), the learning process ends.

[0104] The following is a specific example of the learning process. In the learning process, firstly, the creation unit 121 creates data (input dataset) for learning (steps S101, S102).

[0105] For example, the creation unit 121 extracts input feature data of a length corresponding to the input sequence length from the start frame of the learning time series data, according to a hyperparameter specifying the input sequence length, as input data. The creation unit 121 also extracts predicted feature data of the target time period length from the learning time series data, starting from the next frame of the extracted input data, as forward solution data, according to a hyperparameter specifying the target time period length. The creation unit 121 then creates an input dataset including the extracted input data and the extracted forward solution data.

[0106] The creation unit 121 repeats the above operation while staggering certain time intervals to create multiple input datasets. The certain time interval can be arbitrarily determined, for example, the time interval (frame) for obtaining input feature values.

[0107] This explanation will take the case where the data to be predicted is reservoir inflow, the input series length is 30 days, and the target time period is 3 hours (6 frames) as an example. Assume the time series data used for learning includes... Figure 2 The elements shown, such as the learning period Figure 2 The data shown represents the four years from "2019-01-01 00:30:00 to 2023-01-01 00:00:00".

[0108] In this case, the creation unit 121 extracts data of input features of a length (30 days) corresponding to the length of the input sequence, starting from the start frame, as the initial input data. The initial input data period is the 30 days from "2019-01-01 00:30:00 to 2019-01-31 00:00:00". The input data and... Figure 2This also includes the actual value of reservoir inflow and the actual value of rainfall at 10 locations.

[0109] On the other hand, the creation unit 121 extracts the predicted feature data for the object time period (3 hours) starting from the next frame of the input data as the forward solution data. The time period of the forward solution data corresponding to the initial input data is the 3 hours from "2019-01-31 00:30:00 to 2019-01-31 03:00:00". The forward solution data only includes the actual value of the reservoir inflow.

[0110] With the time frames staggered successively, the next input data period is the 30 days from "2019-01-01 01:00:00 to 2019-01-31 00:30:00", and the correct answer data period is the 3 hours from "2019-01-31 01:00:00 to 2019-01-31 03:30:00".

[0111] This example is equivalent to an example where there is no gap between the input data period and the correct answer data period, i.e., the input period is the period immediately preceding the target period. As mentioned above, a gap can also be left between the input data period and the correct answer data period.

[0112] Additionally, as mentioned above, the periodic variations of the feature quantities can also be considered to determine the input and target time periods. For example, if the time series data has a periodicity on an annual basis, the initial input data period could be the same as described above: "2019-01-01 00:30:00 ~ 2019-01-31 00:00:00" (30 days). On the other hand, regarding the period of the positive solution data, since it is set to the length of the target time period (3 hours) starting from the frame corresponding to one year after the input data, it could be, for example, "2020-01-01 00:30:00 ~ 2020-01-01 03:00:00". In this way, the input and target time periods can be set to a distance with periodic intervals.

[0113] In this way, input data and forward retrieval data are created sequentially for each time frame. Therefore, the number of input datasets is (number of days for learning with time series data - number of days in the input sequence length) × (number of time frames per day). In the example above,

[0114] • Number of days spent learning to use time series data = 1461

[0115] • Input sequence length in days = 30

[0116] • Number of time frames per day = 48

[0117] Therefore, the number of input datasets is (1461-30)×48=68688.

[0118] The input dataset can be represented in any data format, for example, it can be represented by a three-dimensional tensor as shown below. For example, creation unit 121 connects multiple input datasets based on normalization and standardization of each feature quantity, creating a three-dimensional tensor (number of datasets, number of time frames, number of features). For example, the input data tensor is a three-dimensional tensor (68688, 1440, 11), and the tensor of the positive solution data is a three-dimensional tensor (68688, 6, 1).

[0119] Next, in the learning process, the prediction model 112 using the created input dataset is learned (steps S103~S105).

[0120] The learning method used by the learning unit 122 to learn the prediction model 112 can be any method. For example, the learning unit 122 can use a three-dimensional tensor representing the input dataset to perform mini-batch learning.

[0121] As mentioned above, prediction model 112 can be a model with any architecture. The following mainly illustrates an example using LSTM-based prediction model 112.

[0122] The number of hidden layers in an LSTM can be set to any value; the following explanation uses 300 layers as an example. The LSTM outputs data of the same length as the time series data of the input data and the same number of features as the number of hidden layers.

[0123] For example, assuming a batch size of 70, the LSTM itself transforms the 3D tensor (70, 1440, 11) (input data) into a 3D tensor (70, 1440, 300). Therefore, the LSTM model can be constructed as follows.

[0124] • By extracting a certain time period in the time series direction of the LSTM's own output data (e.g., the last 6 frames = the amount of the object time period), the three-dimensional tensor (70, 1440, 300) is transformed into (70, 6, 300).

[0125] The transformed three-dimensional tensor is further input into the fully connected layer of input layer 300 and output layer 1, so that it outputs a three-dimensional tensor (70, 6, 1) representing a feature quantity whose time series length is the length of the object period and corresponds to the reservoir inflow.

[0126] Regarding the learning method of the learning unit 122 for the prediction model 112, any method that can be applied to the structure of the adopted prediction model 112 can be arbitrary. For example, the learning unit 122 calculates the loss based on the three-dimensional tensor of the output data and the tensor of the forward solution data of the LSTM model, calculates the gradient when the parameters (weights and biases) of the LSTM are changed using the backpropagation method, and updates the parameters using the probabilistic gradient descent method. The optimization method and loss function used during learning can be arbitrary. For example, the optimization method is Adam, and the loss function is L1 loss.

[0127] The learning unit 122 stores the prediction model 112 obtained through learning in the storage unit 110. When learning is performed on the initial input data using the value of random number seed 1, the learning unit 122 stores the prediction model 112 corresponding to the value of random number seed 1 in the storage unit 110.

[0128] The learning control unit 120 performs the above operations on each value of the switching object parameter. In the example using the values ​​of 5 random number seeds, the learning control unit 120 stores 5 prediction models 112 corresponding to the values ​​of the 5 random number seeds 1, 3, 5, 7, and 9 in the storage unit 110.

[0129] Next, the prediction processing performed by the information processing device 100 will be described. The prediction processing is the process of predicting the predicted values ​​of the predicted features using the learned prediction model 112. Figure 5 This is a flowchart illustrating an example of the prediction processing in this embodiment.

[0130] The following example illustrates the flowchart of the prediction process.

[0131] • The time period for performing forecast processing: between "2023-01-01 00:00:00" and "2023-01-01 00:30:00".

[0132] Evaluation period: "2023-01-01 00:30:00 ~ 2024-01-01 00:00:00" (this year)

[0133] Input sequence length: 30 days

[0134] • Length of the target period: 3 hours

[0135] Furthermore, it is assumed that the actual values ​​in the evaluation period are unknown at the start of the prediction process. In the prediction process, the predicted values ​​of the predicted features are predicted sequentially for each object period, which is 3 hours in length. It is assumed that the actual value for a given object period within the evaluation period can be obtained after that object period has elapsed. The obtained actual values ​​are used to select a group of values ​​for the switching object parameters to be used in the next object period. The time series data used for prediction (auxiliary feature 115, predicted feature 116) can be of arbitrary length, for example, with... Figure 3 Both are 30 days. Figure 3 In the data, time series data exists starting from “2022-12-02 00:30:00”, which is equivalent to 30 days prior to the start date of the evaluation period.

[0136] The creation unit 131 reads time-series data of auxiliary feature quantity 115 and predicted feature quantity 116 from the storage unit 110 (step S201). Furthermore, after the second repetition, since time-series data of predicted feature quantity 116 was obtained using the selection unit 133 (step S211 described later), the creation unit 131 can use the already obtained predicted feature quantity 116. The creation unit 131 uses the read time-series data to create, for example... Figure 3 The prediction of the structure shown uses time series data (step S202).

[0137] Furthermore, the following scenario envisions using time series data from the immediately preceding input period to predict the predicted values ​​of the target period's predictive features. A gap can be left between the input and target periods, similar to the learning process. Additionally, if the time series data exhibits periodicity, the periodic variations in the feature quantities can be considered to determine the input and target periods.

[0138] The prediction unit 132 reads and sets the switching target parameters (hyperparameter 111) of the unprocessed group from the storage unit 110 (step S203). The prediction unit 132 reads the prediction model 112 corresponding to the set switching target parameters from the storage unit 110, and uses the read prediction model 112 to predict the predicted values ​​of the prediction feature quantities of the target time period (step S204).

[0139] For example, the prediction unit 132 inputs the data of the time series data for the period from "2022-12-02 00:30:00 to 2023-01-01 00:00:00" into the prediction model 112 as input data, and calculates the predicted value for the target time period. The prediction unit 132 stores the predicted value in, for example, the storage unit 110 (step S205).

[0140] The prediction unit 132 determines whether all groups of values ​​for the switching object parameters have been processed (step S206). If not, it returns to step S203 and repeats the processing for the next unprocessed group.

[0141] When all groups of values ​​for switching object parameters have been processed (step S206: Yes), in order to select and output the final predicted value from the multiple predicted values ​​corresponding to the groups of values ​​for multiple switching object parameters respectively, the following processing is performed.

[0142] Furthermore, at this point in time, multiple predicted values ​​output from multiple prediction models 112 corresponding to groups of values ​​of multiple switching object parameters are stored in storage unit 110, etc. In the example using the values ​​of 5 random number seeds, the 5 predicted values ​​corresponding to the values ​​of the 5 random number seeds 1, 3, 5, 7, and 9 are stored in storage unit 110.

[0143] Figure 6 This is a diagram illustrating an example of a data structure for storing predicted values. (Example:) Figure 6 As shown, the prediction unit 132 stores the predicted values ​​of the predicted feature quantity (reservoir inflow) of each frame and each switching object parameter contained in the object time period in the storage unit 110.

[0144] return Figure 5 The following is an explanation. First, the prediction unit 132 determines whether the optimal value of the switching object parameter is stored in the storage unit 110 (step S207). In the first repeated processing, that is, the processing for the first object time period within the evaluation period, since the selection process of the optimal value of the switching object parameter by the selection unit 133 is not performed (step S213, etc., described later), the optimal value is not stored.

[0145] If the optimal value is not stored in the storage unit 110 (step S207: No), the prediction unit 132 uses multiple prediction values ​​corresponding to the group of values ​​of multiple switching object parameters to calculate and output the final prediction data.

[0146] For example, prediction unit 132 calculates a simple average (simple ensemble average) of multiple predicted values ​​and outputs the calculation result as the final predicted data. Prediction unit 132 may also calculate a weighted average (weighted ensemble average) of multiple predicted values ​​based on predetermined weights as the final predicted data. The weights can be determined in any way. For example, a portion of the learning period used for learning prediction model 112 can be designated as a validation period, and the weights can be determined based on the prediction results during the validation period.

[0147] When the optimal value is stored in the storage unit 110 (step S207: Yes), the prediction unit 132 outputs the predicted value for the switching object parameter corresponding to the optimal value as the final prediction data (step S209).

[0148] The prediction control unit 130 determines whether to end the prediction (step S210). For example, the prediction control unit 130 determines to end the prediction when the target time period has reached the end time of the evaluation period. When the prediction is determined to end (step S210: Yes), the prediction process ends.

[0149] When it is determined that the prediction should not be terminated (step S210: No), the selection unit 133 acquires the time series data (actual value) of the predicted feature quantity of the target time period (step S211).

[0150] For example, assuming that a prediction has been performed on the first target time period (2023-01-01 00:30:00~2023-01-01 03:00:00) (step S208), at a certain time in the period "2023-01-01 03:00:00~2023-01-01 03:30:00", which is the time period after the first target time period, time series data (auxiliary feature 115, prediction feature 116) including the actual value of the predicted feature quantity (reservoir inflow) in the first target time period is added to the storage unit 110.

[0151] In this case, the selection unit 133 obtains the actual value of the reservoir inflow for the first target time period. Additionally, the selection unit 133 obtains multiple predicted values ​​for multiple switching target parameters (step S212). For example, the selection unit 133 reads the predicted values ​​of each of the multiple switching target parameters stored in step S205 from the storage unit 110.

[0152] The selection unit 133 calculates an evaluation value based on the difference between each of the multiple predicted values ​​and the actual values, and uses the calculated evaluation value to select a group of more than one optimal hyperparameter values ​​(optimal values) (step S213). Then, it returns to step S201 and repeats the processing for the next target time period.

[0153] Here, it is assumed that the error (mean absolute error, etc.) is used as the evaluation value, and the error of the prediction value of the prediction model 112 corresponding to the random number seed 5 is minimized. In this case, the selection unit 133 stores the value of the switching object parameter, the random number seed 5, as the optimal value in the storage unit 110.

[0154] The processing for the next target time period (the second target time period, 2023-01-01 03:30:00~2023-01-01 06:00:00) will be explained. Assume that at some point during the time period "2023-01-01 03:00:00~2023-01-01 03:30:00", which is the time period following the first target time period, the time series data of the auxiliary feature quantity 115 and the predicted feature quantity 116 of the first target time period are appended to the storage unit 110.

[0155] The creation unit 131 reads the time series data of the additional auxiliary feature quantity 115 and the prediction feature quantity 116 from the storage unit 110 (step S201). The creation unit 131 uses the read time series data to create prediction time series data (step S202).

[0156] For example, the creation unit 131 adds the time series data of the additional feature quantity to the prediction time series data created so far, thereby creating (updating) the prediction time series data. In the processing of the second target time period, the time period of the prediction time series data exists up to "2022-12-02 00:30:00~2023-01-01 03:00:00".

[0157] Similar to the processing for the first target time period, the prediction unit 132 uses the created prediction time series data, switching target parameters and prediction model 112 to predict the next target time period (steps S203 to S206).

[0158] In this example, data from the 30-day period from "2022-12-02 03:30:00 to 2023-01-01 03:00:00" of the predicted time series data is used as input data. For the next target period, predicted values ​​are also calculated for each group of switching target parameters (e.g., 5 random number seeds 1, 3, 5, 7, 9), and the calculated predicted values ​​are stored in storage unit 110.

[0159] In the second target time period, the storage unit 110 stores the optimal value of the switching target parameter selected in the immediately preceding target time period (the first target time period) (step S207: Yes). Therefore, the prediction unit 132 outputs the predicted value of the switching target parameter corresponding to the stored optimal value as the final prediction data (step S209). In the above example, the predicted value corresponding to the random number seed 5 is output as the final prediction data.

[0160] When multiple switching object parameters are selected, as described above, the final prediction data can be output as a simple average or weighted average of the multiple predicted values ​​corresponding to the multiple switching object parameters.

[0161] The same process is repeated for subsequent object time periods (from the 3rd to the 4th object time period) until the end of the evaluation period.

[0162] In this way, the information processing apparatus of this embodiment selects and outputs the prediction value obtained by the prediction model corresponding to the value of the hyperparameter that yielded the best prediction value in the previous prediction among multiple prediction values ​​obtained by multiple prediction models corresponding to different hyperparameter values. As a result, the accuracy of the prediction using the model can be improved.

[0163] Next, use Figure 7 The hardware structure of the information processing device in the embodiments will be described. Figure 7 This is an explanatory diagram showing an example of the hardware structure of an information processing device according to an embodiment.

[0164] The information processing apparatus of the embodiment includes a control device such as a CPU (Central Processing Unit) 51, storage devices such as ROM (Read Only Memory) 52 and RAM (Random Access Memory) 53, a communication I / F 54 connected to a network for communication, and a bus 61 connecting the various parts.

[0165] The program executed by the information processing device of the embodiment is pre-programmed into ROM 52 and the like.

[0166] The program executed by the information processing device of the embodiment may also be configured to be provided as a computer program product as a file recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), floppy disk (FD), CD-R (Compact Disk Recordable), and DVD (Digital Versatile Disk).

[0167] Furthermore, the program executed by the information processing device of the embodiment can be stored on a computer connected to a network such as the Internet and provided via network download. Alternatively, the program executed by the information processing device of the embodiment can be provided or distributed via a network such as the Internet.

[0168] The program executed by the information processing apparatus of the embodiment enables the computer to function as a component of the aforementioned information processing apparatus. In this computer, the CPU 51 is capable of reading the program from a computer-readable storage medium and executing it on the main storage device.

[0169] The following describes a structural example of the implementation method.

[0170] (Structure Example 1)

[0171] An information processing apparatus includes a processing unit that performs the following processing:

[0172] For multiple models, the input feature quantities obtained in the first input time period are input into each model, and the first inferred value of the object feature quantity in the first object time period after the first input time period is calculated from the output of the model. The multiple models correspond to multiple groups, each containing values ​​of one or more hyperparameters, where at least some of the values ​​of the hyperparameters are different from each other. These are multiple models learned using the values ​​of the hyperparameters contained in the corresponding groups.

[0173] Using an evaluation value based on the difference between the first predicted value obtained using the corresponding model and the actual value of the object feature quantity obtained in the first object time period, one or more of the groups are selected from the plurality of groups.

[0174] For each of the multiple models, the input feature quantities obtained in the second input period after the first input period are input into the model, and the second predicted value of the object feature quantity in the second object period after the second input period is calculated from the output of the model.

[0175] The output is one or more second predicted values ​​from one or more models corresponding to one or more selected groups.

[0176] (Structure Example 2)

[0177] According to the information processing apparatus described in Structural Example 1, wherein...

[0178] The more than one hyperparameters include multiple random number seeds used during the learning of the model.

[0179] (Structure Example 3)

[0180] According to the information processing apparatus described in structural example 1 or 2, wherein,

[0181] The first input time period is the time period immediately preceding the first target time period.

[0182] The second input time period is the time period immediately preceding the second object time period.

[0183] (Structure Example 4)

[0184] According to the information processing apparatus described in structural example 1 or 2, wherein,

[0185] The object's characteristic quantity is a characteristic quantity that changes at regular intervals.

[0186] The first input time period is a time period corresponding to the first object time period that is included in the period preceding the period including the first object time period.

[0187] The second input time period is a time period that corresponds to the second object time period and is included in the period preceding the period including the second object time period.

[0188] (Structure Example 5)

[0189] The information processing apparatus according to any one of structural examples 1 to 4, wherein,

[0190] The evaluation value is based on the mean absolute error, mean absolute percentage error, mean absolute scaling error, mean square error, root mean square error, continuous sorting probability score, or coefficient of determination between the first predicted value and the actual value.

[0191] (Structure Example 6)

[0192] The information processing apparatus according to any one of structural examples 1 to 5, wherein,

[0193] The processing unit:

[0194] Use the evaluation value to select multiple groups.

[0195] Calculate multiple second inferred values ​​from multiple models corresponding to the selected multiple groups, and output the values ​​based on the multiple second inferred values ​​as inferred values ​​of the object feature quantity in the second object time period.

[0196] (Structure Example 7)

[0197] According to the information processing apparatus described in Structural Example 6, wherein...

[0198] The processing unit:

[0199] Calculate the weights assigned to each of the multiple second inferred values.

[0200] The weighted average of the multiple second inferred values ​​obtained using the calculated weights is output as the inferred value of the object feature quantity in the second object time period.

[0201] (Structure Example 8)

[0202] According to the information processing apparatus described in Structural Example 6, wherein...

[0203] The processing unit outputs a simple average of the second inferred values ​​from the outputs of the multiple models corresponding to the selected multiple groups as an inferred value of the object feature quantity in the second object time period.

[0204] (Structure Example 9)

[0205] The information processing apparatus according to any one of structural examples 1 to 8, wherein,

[0206] The processing unit calculates a simple average or weighted average of the multiple first inferred values ​​output from the multiple models.

[0207] (Structure Example 10)

[0208] An information processing method, executed by an information processing device, includes:

[0209] For multiple models, the steps are as follows: inputting the input feature values ​​obtained in the first input time period into the model respectively, and calculating the first inferred value of the object feature value in the first object time period after the first input time period output by the model, wherein the multiple models correspond to multiple groups, each containing the values ​​of one or more hyperparameters, and at least some of the values ​​of the one or more hyperparameters are different from each other, and are multiple models learned using the values ​​of the hyperparameters contained in the corresponding group;

[0210] The step of selecting one or more of the groups from a plurality of groups using an evaluation value based on the difference between the first inferred value obtained using the corresponding model and the actual value of the object feature quantity obtained in the first object time period;

[0211] The steps are as follows: for each of the multiple models, the input feature quantities obtained in the second input period after the first input period are input into the model, and the second inferred value of the object feature quantity in the second object period after the second input period is calculated from the output of the model; and

[0212] The step of outputting one or more second predicted values ​​from one or more models corresponding to one or more selected groups.

[0213] (Structure Example 11)

[0214] A program used to make a computer execute:

[0215] For multiple models, the steps are as follows: inputting the input feature values ​​obtained in the first input time period into the model respectively, and calculating the first inferred value of the object feature value in the first object time period after the first input time period output by the model, wherein the multiple models correspond to multiple groups, each containing the values ​​of one or more hyperparameters, and at least some of the values ​​of the one or more hyperparameters are different from each other, and are multiple models learned using the values ​​of the hyperparameters contained in the corresponding group;

[0216] The step of selecting one or more of the groups from a plurality of groups using an evaluation value based on the difference between the first inferred value obtained using the corresponding model and the actual value of the object feature quantity obtained in the first object time period;

[0217] The steps are as follows: for each of the multiple models, the input feature quantities obtained in the second input period after the first input period are input into the model, and the second inferred value of the object feature quantity in the second object period after the second input period is calculated from the output of the model; and

[0218] The step of outputting one or more second predicted values ​​from one or more models corresponding to one or more selected groups.

[0219] While several embodiments of the invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, with various omissions, substitutions, and modifications possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, and are included within the scope of the invention as set forth in the claims and its equivalents.

Claims

1. An information processing apparatus comprising a processing unit, the processing unit performing the following processing: For multiple models, the input feature quantities obtained in the first input time period are input into each model, and the first inferred value of the object feature quantity in the first object time period after the first input time period is calculated from the output of the model. The multiple models correspond to multiple groups, each containing values ​​of one or more hyperparameters, where at least some of the values ​​of the hyperparameters are different from each other. These are multiple models learned using the values ​​of the hyperparameters contained in the corresponding groups. Using an evaluation value based on the difference between the first predicted value obtained using the corresponding model and the actual value of the object feature quantity obtained in the first object time period, one or more of the groups are selected from the plurality of groups. For multiple models, the input feature quantities obtained in the second input period after the first input period are input into the model respectively, and the second predicted value of the object feature quantity in the second object period after the second input period is calculated from the output of the model. The output is one or more second predicted values ​​from one or more models corresponding to one or more selected groups.

2. The information processing apparatus according to claim 1, wherein, The more than one hyperparameters include multiple random number seeds used during the learning of the model.

3. The information processing apparatus according to claim 1 or 2, wherein, The first input time period is the time period immediately preceding the first target time period. The second input time period is the time period immediately preceding the second object time period.

4. The information processing apparatus according to claim 1 or 2, wherein, The object's characteristic quantity is a characteristic quantity that changes at regular intervals. The first input time period is a time period corresponding to the first object time period that is included in the period preceding the period including the first object time period. The second input time period is a time period that corresponds to the second object time period and is included in the period preceding the period including the second object time period.

5. The information processing apparatus according to any one of claims 1 to 4, wherein, The evaluation value is based on the mean absolute error, mean absolute percentage error, mean absolute scaling error, mean square error, root mean square error, continuous sorting probability score, or coefficient of determination between the first predicted value and the actual value.

6. The information processing apparatus according to any one of claims 1 to 5, wherein, The processing unit: Use the evaluation value to select multiple groups. Calculate multiple second inferred values ​​from multiple models corresponding to the selected multiple groups, and output the values ​​based on the multiple second inferred values ​​as inferred values ​​of the object feature quantity in the second object time period.

7. The information processing apparatus according to claim 6, wherein, The processing unit: Calculate the weights assigned to each of the multiple second inferred values. The weighted average of the multiple second inferred values ​​obtained using the calculated weights is output as the inferred value of the object feature quantity in the second object time period.

8. The information processing apparatus according to claim 6, wherein, The processing unit outputs a simple average of the second inferred values ​​from the outputs of the multiple models corresponding to the selected multiple groups as an inferred value of the object feature quantity in the second object time period.

9. The information processing apparatus according to any one of claims 1 to 8, wherein, The processing unit calculates a simple average or weighted average of the multiple first predicted values ​​output from the multiple models.

10. An information processing method, executed by an information processing device, comprising: For multiple models, the steps are as follows: inputting the input feature quantity obtained in the first input time period into the model respectively, and calculating the first inferred value of the object feature quantity in the first object time period after the first input time period output by the model, wherein the multiple models correspond to multiple groups that each contain the values ​​of one or more hyperparameters and at least some of the values ​​of the one or more hyperparameters are different from each other, and are multiple models learned using the values ​​of the hyperparameters contained in the corresponding groups; The step of selecting one or more of the groups from a plurality of groups using an evaluation value based on the difference between the first inferred value obtained using the corresponding model and the actual value of the object feature quantity obtained in the first object time period; For multiple models, the input feature quantities obtained in the second input period after the first input period are input into the model respectively, and the second inferred value of the object feature quantity in the second object period after the second input period is calculated from the output of the model; as well as The step of outputting one or more second predicted values ​​from one or more models corresponding to one or more selected groups.

11. A storage medium storing a program for causing a computer to execute: For multiple models, the steps are as follows: inputting the input feature quantity obtained in the first input time period into the model respectively, and calculating the first inferred value of the object feature quantity in the first object time period after the first input time period output by the model, wherein the multiple models correspond to multiple groups that each contain the values ​​of one or more hyperparameters and at least some of the values ​​of the one or more hyperparameters are different from each other, and are multiple models learned using the values ​​of the hyperparameters contained in the corresponding groups; The step of selecting one or more of the groups from a plurality of groups using an evaluation value based on the difference between the first inferred value obtained using the corresponding model and the actual value of the object feature quantity obtained in the first object time period; For multiple models, the input feature quantities obtained in the second input period after the first input period are input into the model respectively, and the second inferred value of the object feature quantity in the second object period after the second input period is calculated from the output of the model; as well as The step of outputting one or more second predicted values ​​from one or more models corresponding to one or more selected groups.

Images