Information processing systems, information processing methods, programs

Abstract

The present invention aims to provide a mechanism for improving forecasting accuracy in demand forecasting. [Solution] The present invention is characterized by comprising: a selection means that uses time series data relating to the target of prediction and a prediction model for calculating future predicted values ​​to calculate the error between the predicted values ​​based on the prediction model and the time series data, and selects a plurality of prediction models based on the error; and a calculation means that uses the plurality of prediction models to calculate future predicted values ​​relating to the target of prediction.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Improving prediction accuracy in demand prediction systems is one of the major challenges, and various effects such as inventory cost reduction, stockout risk reduction, and logistics cost reduction can be obtained by improving the accuracy. In recent years when the shortage of logistics resources has become a social issue, the importance of improving prediction accuracy is increasing more and more.

[0003] For example, Patent Document 1 discloses a model acquisition unit that acquires a single-item prediction model for predicting the demand of a target product and a category prediction model for predicting the demand of a category including the target product, a calculation unit that calculates the accuracy of demand prediction for the target product in the single-item prediction model and the category prediction model, and an adoption unit that adopts either the single-item prediction model or the category prediction model based on the accuracy.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0005] Accordingly, an object of the present invention is to provide a mechanism for improving prediction accuracy in demand prediction.

Means for Solving the Problems

[0006] The present invention is characterized by comprising: selection means for calculating the error between predicted values ​​based on a prediction model and the time series data, using time series data relating to the target of prediction and a prediction model for calculating future predicted values, and selecting a plurality of prediction models based on the error; and calculation means for calculating future predicted values ​​relating to the target of prediction using the plurality of prediction models. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a mechanism for improving the forecasting accuracy in demand forecasting. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the configuration of a supply and demand management system in an embodiment of the present invention. [Figure 2] This figure shows an example of the hardware configuration of a client terminal 110 and a server 130 in an embodiment of the present invention. [Figure 3] This figure shows an example of the process outline in an embodiment of the present invention. [Figure 4] This figure shows an example of an AI-DFC model master 400 and an AI-DFC model attribute definition master 410 in an embodiment of the present invention. [Figure 5] This figure shows an example of an AI-DFC statistical method calculation definition master 500 and an AI-DFC model candidate master 510 in an embodiment of the present invention. [Figure 6] This figure shows an example of an item site master 600 in an embodiment of the present invention. [Figure 7] This figure shows an example of a flowchart of the mixing ratio calculation process performed by the information processing device in an embodiment of the present invention. [Figure 8] This figure shows an example of AI-DFC model mixing ratio data 800 in an embodiment of the present invention. [Figure 9]This is a diagram illustrating an example of a flowchart of learning data output processing executed by an information processing apparatus according to an embodiment of the present invention. [Figure 10] This is a diagram illustrating an example of learning data 1000 according to an embodiment of the present invention. [Figure 11] This is a diagram illustrating an example of a flowchart of prediction result calculation processing executed by an information processing apparatus according to an embodiment of the present invention. [Figure 12] This is a diagram illustrating an example of prediction result calculation processing executed by an information processing apparatus according to an embodiment of the present invention. [Figure 13] This is a diagram illustrating an example of shipping performance data 1300 according to an embodiment of the present invention. [Figure 14] This is a diagram illustrating an example of a demand characteristic classification master 1400 according to an embodiment of the present invention. [Mode for Carrying Out the Invention]

[0009] (First Embodiment) Figure 1 is a diagram illustrating an example of the system configuration of a supply and demand management system 100 according to an embodiment of the present invention.

[0010] The configuration is such that a client terminal 110 and a server 130 are connected via a network 120.

[0011] The client terminal 110 displays a screen transmitted from the server 130, receives data input from a user, and transmits the input data to the server 130.

[0012] The server 130 receives an execution request from the client terminal 110, executes processing, generates a screen for displaying the result, and transmits the screen to the client terminal 101.

[0013] The network 120 is the Internet, a LAN (LOCAL AREA NETWORK), or the like.

[0014] Figure 2 is a block diagram showing an example of a hardware configuration of an information processing apparatus applicable as the server 130 and the client terminal 110 of the present invention.

[0015] As shown in Figure 2, in the information processing apparatus, a CPU (CENTRAL PROCESSING UNIT) 201, a RAM (RANDOM ACCESS MEMORY) 202, a ROM (READ ONLY MEMORY) 203, an input controller 205, a video controller 206, a memory controller 207, and a communication I / F controller 208 are connected via a system bus 204.

[0016] The CPU 201 comprehensively controls each device and controller connected to the system bus 204.

[0017] The RAM 202 functions as the main memory, work area, and the like of the CPU 201. The CPU 201 loads programs and the like required for executing processing from the ROM 203 or the external memory 211 to the RAM 202, and executes the loaded programs to implement various operations.

[0018] The ROM 203 or the external memory 211 holds a BIOS (BASIC INPUT / OUTPUT SYSTEM) and an OS (OPERATING SYSTEM), which are control programs executed by the CPU 201, as well as computer-readable and executable programs for implementing the present information processing method and various necessary data (including data tables).

[0019] The input controller 205 controls inputs from input devices such as a keyboard 209 and pointing devices such as a mouse (not shown). When the input device is a touch panel, a user can give various instructions by pressing (touching with a finger or the like) in alignment with icons, cursors, and buttons displayed on the touch panel.

[0020] Furthermore, the touch panel may be a multi-touch screen or other touch panel capable of detecting the positions of multiple fingers touching it.

[0021] The video controller 206 controls the display to an external output device such as the display 210. The display includes the display of a notebook computer integrated with the main unit. The external output device is not limited to a display; for example, it may be a projector. Furthermore, for the aforementioned touch-enabled device, an input device is also provided.

[0022] The video controller 206 can control the video memory (VRAM) used for display control. It can use a portion of RAM 202 as the video memory area, or it can provide a separate, dedicated video memory.

[0023] The memory controller 207 controls access to the external memory 211. The external memory can include an external storage device (hard disk), a flexible disk (FD), or a CompactFlash® memory connected to a PCMCIA card slot via an adapter, which stores boot programs, various applications, font data, user files, editing files, and other data.

[0024] The communication interface controller 208 connects to and communicates with external devices via a network and performs communication control processing over the network. For example, it can communicate using TCP / IP, telephone lines such as ISDN, and 3G mobile phone lines.

[0025] Furthermore, the CPU 201 enables display on the display 210 by, for example, performing the process of expanding (rasterizing) outline fonts into the display information area in RAM 202. The CPU 201 also enables user input via a mouse cursor (not shown) on the display 210.

[0026] Figure 3 shows an example of the processing overview in an embodiment of the present invention. The processing flow will be explained in two stages: (1) the learning stage and (2) the prediction stage.

[0027] (1) Regarding the learning stage During the learning phase, (a) attribute information of the product and (b) demand characteristics from historical demand data are calculated and used as explanatory variables. Furthermore, the trained model is generated using the (d) optimal time series forecast model data selected by the (c) optimal time series forecast model selection process described later as the target variable.

[0028] First, the (a) attribute information definitions set for each model in the Model Attribute Definition Master 410 are obtained. Based on the defined attribute information, attribute information is obtained from the product database, etc. The attribute information can be anything, and may include, for example, product classification, characteristics, sales region, sales period, etc. Multiple attributes may be set for each model, for example, attributes such as product classification and warehouse location may be combined and set for each SKU (stock keeping unit).

[0029] Next, demand characteristics are calculated from the shipment performance data 1300 (described later in Figure 13) to obtain (b) demand characteristics data. Demand characteristics are statistical values ​​calculated from the shipment performance data 1300. Therefore, the calculation of demand characteristics corresponds to the statistical calculation process (step S908 in Figure 9). The indicators for the statistics can be set for each model in the statistical calculation method definition master 500, and for example, the mean, standard deviation, correlation coefficient, etc. are used, and the values ​​of each indicator during the evaluation period of the shipment performance data are calculated as (b) demand characteristics data.

[0030] Furthermore, using the shipment performance data 1300 and the evaluation indicators set in the model master table 400 (described later in Figure 4), the optimal time series forecasting model is selected from the models set in the model candidate master 510 (described later in Figure 5). This process is called (c) optimal time series forecasting model selection process, and it is a mechanism that calculates the forecast error over the evaluation period of multiple forecasting models based on past demand performance data and selects the model with the smallest forecast error. Through the above process, (d) optimal time series forecasting model data is obtained for each shipment performance data.

[0031] In the second embodiment described later, (c) the prediction error calculated in the optimal time series prediction model selection process is used to determine the prediction model to be used to predict future predicted values.

[0032] In the first embodiment, the aforementioned (a) attribute information, (b) demand characteristics, and (d) optimal time series forecast model data are used as training data 1000 (described later in Figure 10) to generate a trained model. This completes the processing at the training stage.

[0033] (2) Regarding the prediction stage In the forecasting stage, demand characteristics calculated from actual shipment data of the product to be forecasted, along with product attribute information, are input into the trained model generated in (1) to calculate candidate time series forecasting models and their respective mixing ratios (weights).

[0034] Each statistical value set in the AI-DFC model statistical calculation method definition master (described later in Figure 5) is calculated from the shipment performance data of the target product and used as demand characteristic data. Furthermore, attribute information for the attribute information items set for each model in the model attribute definition master 410 is obtained from the database of the target product, etc.

[0035] By inputting demand characteristic information and attribute information into the trained model, candidate time series forecasting models and the mixing ratio of each time series forecasting model are output. From the model candidates output by the trained model, a model to be used for the forecast value calculation process is selected from the forecasting model candidates using the forecasting model selection method set in the AI-DFC model master table 400 (described later in Figure 12). Furthermore, the mixing ratios of the selected models used for the forecast value calculation process are adjusted so that the sum of the mixing ratios is 1, and output as AI-DFC model mixing ratio data 800 (described later in Figure 8). This process constitutes the mixing ratio calculation process, and details will be described later using Figure 7.

[0036] The AI-DFC model mixing ratio data is used to calculate the predicted value, along with the model used in the prediction calculation process, the mixing ratio of that model, and past shipment performance data. The predicted value is the future demand for the product being predicted.

[0037] In other words, this is an example of a process that uses multiple forecasting models to calculate the future demand for a product related to the forecast.

[0038] This process corresponds to the predicted value calculation process explained in Figure 11. The above describes the process at the prediction stage.

[0039] First, we will explain an example of the master data used in the mixing ratio calculation process (Figure 7), the training data output process (Figure 9), and the predicted value calculation process (Figure 11) using Figures 4 to 8.

[0040] The AI-DFC model master shown in Figure 4(a) is a master that sets the selection method for selecting the model to be used to calculate the predicted value from the candidate models set for each group (AI-DFC model ID) that contains each model.

[0041] The AI-DFC model master 400 includes the following items: AI-DFC model ID, prediction model selection category, prediction model limit / cumulative ratio, start index, end index, evaluation period, and prediction error ID.

[0042] The AI-DFC model ID is an arbitrary ID assigned to a group containing each model candidate. The AI-DFC model ID may be set according to the characteristics of each product (e.g., regular demand products, seasonal products), or it may be set according to the group or type of product. Alternatively, the same ID may be assigned to all products.

[0043] For example, by assigning an AI-DFC model ID to each of the product characteristics mentioned above, it is possible to set models that are more likely to be selected for each characteristic, such as steady-demand products or seasonal products, as model candidates.

[0044] The prediction model selection category allows you to specify how candidate models are selected, and this can be set using specific numerical criteria such as the upper limit of prediction models / cumulative ratio. For example, if the prediction model selection category is "cumulative ratio" and the cumulative ratio is set to 80% (=0.8), then in the model selection process for calculating the mixed ratio shown in Figure 7, models whose cumulative ratio, calculated by summing the mixed ratios of the top models first, is 0.8 or less will be selected.

[0045] In other words, this is an example of a process that selects a prediction model to be used to calculate the predicted value by the calculation means, based on conditions derived from the weights of the prediction model.

[0046] In other words, this is an example of a process in which, among multiple prediction models, prediction models with the largest weights are selected in order until the sum of their weights satisfies a predetermined condition.

[0047] Furthermore, for example, if the prediction model selection category is set to "Top Specified Number" and the upper limit for the number of prediction models is set to 2, then the two models with the highest mixing ratios will be selected in order.

[0048] In other words, this is an example of a process that selects a predetermined number of prediction models from among multiple prediction models, starting with the prediction models with the largest weights.

[0049] Alternatively, you could select models with a mixing ratio above a certain value. In that case, for example, you could set the prediction model selection category to "Mixing Ratio" and set the value to 0.8, allowing you to select models with a mixing ratio of 0.8 or higher.

[0050] In other words, this is an example of a process in which a prediction model whose weights satisfy predetermined conditions is selected as the prediction model to be used to calculate the predicted value.

[0051] Alternatively, you could select a model that exhibits a mixing ratio greater than or equal to P% of the highest mixing ratio value. For example, if P=20% and the highest mixing ratio value is 0.8, then another model with a mixing ratio of 0.16 or higher would be selected.

[0052] In other words, this is an example of a process in which, among multiple prediction models, the prediction model that satisfies predetermined conditions based on the weight of the prediction model with the largest weight is selected as the prediction model to be used to calculate the predicted value.

[0053] By allowing users to set the selection method and numerical criteria for predictive models, it becomes possible to set a model selection method that is appropriate for each model and product group, which broadens the scope of trial and error and is expected to improve predictive accuracy.

[0054] The start index and end index can each be configured to specify the cycle (period) to be predicted. For example, if start index 1 and end index 1 are set, only the current cycle for which the prediction is being made will be included in the prediction. For example, if start index 1 and end index 3 are set, the period from the current cycle to two cycles after the current cycle for which the prediction is being made will be included in the prediction.

[0055] The evaluation period can be set to the period used to evaluate the model. In this embodiment, it is set in months, but it may also be possible to set it in weeks or days.

[0056] The prediction error ID allows you to specify the evaluation metrics used to evaluate the model. For example, you can set the mean squared error (RMSE) or the mean absolute error (MAE). Using the evaluation metrics set here, the optimal time series prediction model selection process shown in Figure 1(c) is performed. Any metric that can evaluate error can be used as the evaluation metric, and you may also set other metrics such as the mean rate of error (MPE) or mean absolute rate of error (MAPE).

[0057] Figure 4(b) shows an example of the AI-DFC model attribute definition master 410. The AI-DFC model attribute definition master shown in Figure 4(b) is a master that sets the model attributes for each model.

[0058] The AI-DFC model attribute definition master includes fields for AI-DFC data category and AI-DFC attribute name.

[0059] The AI-DFC data classification allows for the registration of each model assigned to an AI-DFC model ID, distinguishing whether it is for production or training. This makes it easier to adjust models; for example, models registered for training validation can be used for verification, and models that perform well in training validation can be re-registered for production use, facilitating trial and error.

[0060] AI-DFC attribute names allow you to set the names of the attributes used in each model. Attributes can include, for example, product attributes or the region where the product is sold. For instance, if a user determines that the selected model may change depending on the region where the product is sold, including the region as an attribute allows the trained model to consider regional characteristics when selecting a model.

[0061] Figure 5(a) shows an example of the AI-DFC statistical calculation method definition master 500. The AI-DFC statistical calculation method definition master shown in Figure 5(a) is a data table in which statistical calculation methods can be registered for each model, and includes items such as AI-DFC model ID, AI-DFC data category, statistical ID, and calculation start / end index.

[0062] The statistical ID allows you to select one or more statistical calculation methods for each model: mean, standard deviation, and correlation coefficient. For example, if you select the mean, the mean of the actual demand for the period in question will be used as the statistical value; if you select the standard deviation, the standard deviation of the actual demand for the period in question will be used as the statistical value. This allows users to set statistical indicators that can appropriately represent the characteristics of the time series data.

[0063] The "Calculation Start Index" and "Calculation End Index" fields allow you to specify the start and end cycles for the period during which statistics are calculated. The mean, standard deviation, and correlation coefficient for the period specified in these fields will be calculated as statistics.

[0064] Figure 5(b) shows an example of the AI-DFC model candidate master 510. The AI-DFC model candidate master shown in Figure 5(b) allows setting model parameters for each model candidate and includes fields for AI-DFC model ID, AI-DFC data category, prediction model parameter number, and prediction model parameter name.

[0065] The prediction model parameter number is an arbitrary number assigned to each candidate model. The prediction model parameter name is the specific model parameter name for each candidate model. Various prediction and estimation methods can be set as model parameters. For example, prediction methods such as exponential smoothing, Winters method, and moving average method can be set as model candidates. In addition, autoregressive models and other methods may be used, and any model that can be used for prediction can be set.

[0066] Figure 6 shows an example of the Item Site Master 600. The Item Site Master allows you to set a prediction method for each item (product or SKU) and includes fields for Item Site, Prediction Calculation Method Category, AI-DFC Model ID, Validation AI-DFC Model ID, and Specified Prediction Model Parameter Name.

[0067] Item sites are numbers assigned to each product or group of products, and can be set by combining item (product) and site (location) numbers.

[0068] In the prediction calculation method category, you can set which prediction calculation method to use for each product. For example, if you specify "AI-DFC mix," the future prediction value will be calculated using the prediction values ​​generated by the AI-generated prediction model candidates and the mixing ratio of each model (details will be explained later in Figure 12).

[0069] If "Specified Model" is selected, the user can set the model to be used for prediction in the "Specified Prediction Model Parameters" section, enabling prediction using the specified model.

[0070] In other words, this is an example of a process that calculates future predicted values ​​related to the target of prediction using a prediction model specified by the user.

[0071] As mentioned above, it is possible to set whether to use a model selected using AI output for each product, or a model specified by the user. This allows users to customize and set the most accurate method for each product's characteristics.

[0072] Figure 7 shows an example of a flowchart of the process performed by the information processing device in an embodiment of the present invention.

[0073] In step S701, the item site master 600 shown in Figure 6 is obtained. From the item site master 600, information on the AI-DFC model ID set for the product to be predicted is obtained.

[0074] In step S702, information from the pre-configured AI-DFC model master 400 shown in Figure 4(a) is acquired. Based on the prediction model selection category and the settings for the upper limit number of prediction models / cumulative ratio of the AI-DFC model master, a model to be used in the prediction value calculation process described later in Figure 11 is selected from the candidate models output by the trained model.

[0075] In other words, step S702 is a step that shows an example of a process for accepting a method for selecting a prediction model from among multiple prediction models to be used for calculating the predicted value by the calculation means. In step S703, information from the pre-configured AI-DFC statistical calculation method definition master 500 shown in Figure 5(a) is acquired. The statistical calculation methods (mean, standard deviation, correlation coefficient, etc.) set for each model are acquired from the AI-DFC statistical calculation method definition master, and in step S708, statistical calculation processing is performed based on the shipment performance data using the acquired statistical calculation methods.

[0076] In step S704, information from the pre-configured AI-DFC model candidate master 510 shown in Figure 5(b) is obtained. From the AI-DFC model candidate master, the model candidates (prediction model parameters) configured for each AI-DFC model ID are obtained.

[0077] In step S705, information from the pre-configured AI-DFC model attribute definition master 410 shown in Figure 4(b) is acquired. Based on the attribute information defined for each AI-DFC model ID in the AI-DFC model attribute definition master, attribute information for each product is acquired.

[0078] Attribute information includes, for example, "item numerical column attribute 1" and "site string attribute 1." For each attribute, the characteristics and values ​​for each product are determined by referring to the product database (not shown) to obtain information about the product and its location. For example, if "item numerical column attribute 1" is information about the product group number (for electrical appliances, this would be copiers, home appliances, audio equipment, etc.), the numerical value related to the product group number is obtained from the product database. For example, if "site string attribute 1" is information about the product's sales region, the string related to the product's sales region ("KANTO", "KANSAI", etc.) is obtained from the product database.

[0079] The processes in steps S701 to S705 described above can be performed in any order, or the information for each master can be acquired all at once.

[0080] In step S706, the steps described later, starting from step S706, are repeated for each AI-DFC model ID.

[0081] In step S707, shipment performance data for the product to be predicted is obtained. Figure 13 shows an example of shipment performance data 1300. Shipment performance data is time-series data that summarizes shipment performance information for each item site cycle. In this embodiment, the data is aggregated every predetermined month, but it may also be aggregated over predetermined periods such as weeks, days, or quarters.

[0082] In step S708, the statistical calculation process for shipment performance is performed based on the shipment performance data of the product to be predicted obtained in step S707 and the information from the AI-DFC statistical calculation method definition master 500 obtained in step S703. For example, if "average" is set as the statistic, the average of the demand performance for the target period is calculated as the statistical value.

[0083] In step S709, the mixing ratio calculation process is executed. In the mixing ratio calculation process, the trained model, which has been pre-trained with the training data shown in Figure 10, is given demand characteristic information, which is a statistical value of the demand characteristics for each product and its group calculated from the shipment performance data calculated in step S708, and attribute information for each product and its group obtained in step S705, to output the model candidate and its mixing ratio.

[0084] Based on the model candidates output by the trained model and their mixing ratios, and the information on the prediction model selection category selected in the AI-DFC model master obtained in step S702, a model is selected. Details will be described later in Figure 12.

[0085] In other words, step S709 is a step that demonstrates an example of a process in which multiple prediction models for calculating future predicted values ​​related to a target are obtained by inputting time series data related to the target to a trained model that has been trained using time series data and a prediction model for calculating future predicted values.

[0086] In step S710, the iterative processing performed for each AI-DFC model ID is terminated.

[0087] In step S711, the mixing ratio of the model selected in step S709 is output, and the AI-DFC model mixing ratio data 800 shown in Figure 8 is output, and this process ends.

[0088] Figure 8 shows an example of AI-DFC model blend ratio data 800. The AI-DFC model blend ratio data includes items such as item site, monthly cycle number, prediction model parameter number, prediction model parameter name, and AI-DFC blend ratio. This data is output in step S711, and for each prediction target cycle of each item site, the prediction model to be used to calculate future prediction values ​​is selected from the candidate prediction models output by the trained model, based on the selection method set in the AI-DFC model master table in Figure 4(a).

[0089] The monthly cycle number identifies the cycle for which the trained model made predictions. The prediction model parameter number and prediction model parameter name are information about the model output by the trained model that was selected as the prediction model to be used to calculate future prediction values ​​from among the candidate prediction models output by the trained model.

[0090] For example, for the item site "C02431-A100", the models "36-month variable M1 exponential smoothing_estimate" and "36-month variable M1 Winters model_estimate (seasonally unconsidered)" are selected, and their respective mixing ratios are output as "0.286" and "0.714". For each item site's forecast cycle, the forecasting model used to calculate future forecast values ​​and the probability of that model being selected are output as the "mixing ratio".

[0091] Figure 9 shows an example of a flowchart for the training data output process performed by the information processing device. The training data output process described in Figure 9 is the process for outputting the training data 1000 shown in Figure 10.

[0092] In step S901, the item site master 600 shown in Figure 6 is obtained. From the item site master 600, information on the AI-DFC model ID set for the product to be trained is obtained.

[0093] In step S902, we obtain shipment data for the products to be used for training.

[0094] In step S903, information from the AI-DFC model master 400 shown in Figure 4(a) is acquired. Using the start index, end index, evaluation period, and prediction error ID information set in the AI-DFC model master, the prediction error calculation process described later is performed in step S910.

[0095] In step S904, information from the pre-configured AI-DFC model candidate master 510 shown in Figure 5(b) is obtained. Model candidates configured for each AI-DFC model ID are obtained from the AI-DFC model candidate master.

[0096] In step S905, information from the pre-configured AI-DFC model attribute definition master 410 shown in Figure 4(b) is acquired. Based on the attribute information defined for each model in the AI-DFC model attribute definition master, attribute information for each product is acquired.

[0097] In step S906, information from the pre-configured AI-DFC statistical calculation method definition master 500 shown in Figure 5(a) is acquired. The statistical calculation methods (mean, standard deviation, correlation coefficient) set for each model are acquired from the AI-DFC statistical calculation method definition master, and in step S908, statistical calculation processing is performed based on the shipment performance data.

[0098] The processes in steps S901 to S906 described above can be performed in any order, or the master information can be retrieved all at once.

[0099] In step S907, the optimal time series forecasting model selection process is initiated for each cycle, selecting the forecasting model with the smallest forecasting error from the candidate forecasting models.

[0100] Step S908 performs statistical calculation processing. The statistical values ​​used are the mean, standard deviation, correlation coefficient, etc., which are set for each model in the AI-DFC statistical calculation method definition master obtained in step S906.

[0101] In step S909, the predicted values ​​for each prediction model are calculated using the prediction model candidates set in the AI-DFC model candidate master. Specifically, by using the prediction model set in the prediction model parameter name field (e.g., exponential smoothing, Winters method, moving average method, etc.) and historical demand data, the historical predicted values ​​when that model is applied to the prediction of the target product can be calculated. Known methods may be used for calculating predicted values ​​from the prediction model and historical demand data.

[0102] In step S910, based on the evaluation period and prediction error ID set in the AI-DFC model master, the prediction error between the prediction results of each prediction model calculated in step S909 and historical demand data is calculated. The prediction model parameters of the model with the smallest prediction error are obtained, and that model is determined as the optimal time series prediction model.

[0103] In this embodiment, the model with the smallest prediction error is selected as the optimal model. However, based on the prediction error results, multiple models may be selected and included in the training data.

[0104] Furthermore, in step S910, multiple models may be selected based on the prediction error, and future prediction values ​​may be calculated using the predicted values ​​of the multiple models and the values ​​based on the prediction error, without using a pre-trained model. This method will be described later as a second embodiment.

[0105] In step S911, the process of selecting the optimal time series forecasting model for each cycle is terminated.

[0106] In step S912, the prediction model parameter number of the model with the smallest prediction error, determined for each cycle, is obtained, and the training data 1000 shown in Figure 10 is output, ending the process.

[0107] Figure 10 shows an example of 1000 training data points. The training data includes, for example, demand structure number, monthly cycle number, forecast model parameter number, attribute information (item string attribute, site string attribute, item numerical column attribute, site numerical column attribute), and statistical values ​​(mean, standard deviation, correlation coefficient).

[0108] The demand structure number is a number that uniquely identifies an SKU, and may be a code assigned to each product or warehouse, for example. The monthly cycle number is a number that identifies the cycle for which the forecast was made.

[0109] The forecast model parameter number is the model selected in the optimal time series forecast model selection process in step S910 for the forecasted cycle in that demand structure number.

[0110] The attribute information can be set as item string attribute, site string attribute, item numeric column attribute, and site numeric column attribute, for example, product attributes or the region where the product is sold. The attribute information to be included in the training data can be set by the user in the AI-DFC model attribute definition master 410. The attribute information is acquired in step S905 and output as training data.

[0111] In this embodiment, attribute information is classified and set into items, sites, strings, and numbers, but it is not limited to this, and it may be registered without classification.

[0112] The statistical values ​​include the mean, standard deviation, and correlation coefficient. These statistical values ​​are calculated from monthly shipment performance data for a specified period prior to the current month (the month in which the forecast was made). In this embodiment, the calculations are performed monthly, but they may also be performed weekly or daily. The statistical information is calculated in step S908 and output as training data.

[0113] In this embodiment, pre-trained models are generated for each product characteristic (e.g., steady-state demand products, seasonal products) or demand pattern using training data as shown in Figure 9. However, the model is not limited to this, and other attributes may be used for grouping.

[0114] Figure 11 shows an example of a flowchart for the prediction result calculation process performed by the information processing device.

[0115] In step S1101, the AI-DFC model master shown in Figure 4(a) is obtained, and the settings for the prediction model selection category and the upper limit number of prediction models / cumulative ratio are acquired.

[0116] In step S1102, the AI-DFC model candidate master is obtained, and the parameter numbers and prediction model parameter names of the candidate prediction models for each model are obtained.

[0117] In step S1103, shipping performance data for the item site to be predicted is obtained.

[0118] In step S1104, the mixing ratio data is acquired. This allows us to obtain the model selected by the trained model and selected according to the set selection method.

[0119] In step S1105, the iterative process of calculating the predicted value for each prediction model is started.

[0120] In step S1106, the prediction model parameters selected for the target cycle of the item to be predicted are obtained from the AI-DFC model mixing ratio data shown in Figure 7, and the prediction values ​​are calculated based on past demand data. This process is the same as the process in step S909.

[0121] In step S1107, the iterative process of calculating the predicted value for each prediction model is terminated.

[0122] In step S1108, future prediction values ​​are calculated using the model-specific prediction values ​​calculated in step S1106. The final prediction value is calculated by summing the results of multiplying the prediction value calculated in step S1106 by the AI-DFC mixing ratio value of the AI-DFC model mixing ratio data 800 obtained in step S1104. Details of the calculation method will be described later in Figure 12.

[0123] In other words, step S1108 is a step that demonstrates an example of a process for calculating future predicted values ​​related to the target of prediction using multiple acquired prediction models.

[0124] In other words, step S1108 is a step that demonstrates an example of a process for calculating predicted values ​​using the prediction model identified by the accepted selection method.

[0125] In step S1109, the prediction results are displayed and the process is terminated.

[0126] Figure 12 shows an example of the prediction result calculation process performed by the information processing device in an embodiment of the present invention. First, (1) a method for calculating predicted values ​​using the model candidate output by the trained model and the mixing ratio will be explained ((2) will be explained in the second embodiment).

[0127] Figure 12(1) provides a concrete example of how, in the mixing ratio calculation process of step S709, a model is selected from the model candidates output by the trained model, based on the prediction model selection method selected in the AI-DFC model master 400 acquired in step S702, to be used in the prediction result calculation process shown in Figure 11.

[0128] The output of the trained model, as shown in the left table, outputs the mixing ratio value of each model candidate for each SKU. The trained model, which was trained using product demand characteristics information and attribute information as explanatory variables and the model selected in the optimal time series forecasting model selection process as the objective variable, is given demand characteristics information and attribute information of the product to be predicted, and outputs a suitable forecasting model candidate and its mixing ratio. In other words, the mixing ratio can be said to be the certainty of the model evaluated by the AI, or the probability that the model will be selected. The trained model may output the mixing ratio of all model candidates set in the AI-DFC model candidate master 510, or it may select a suitable model from the model candidates and output the mixing ratio of that model.

[0129] In other words, this is an example of a process in which a trained model outputs multiple prediction models from a set of pre-configured prediction models, based on time-series data related to the prediction target, to calculate future predicted values ​​for that target.

[0130] Based on the output results of the trained model, the model to be used to calculate the predicted value is selected based on the selection method set in the AI-DFC model master 400, and output as AI-DFC model mixture ratio data 800.

[0131] For example, in AI-DFC Model Master 400, if the prediction model selection category is set to "Cumulative Ratio" and the cumulative ratio value is set to "0.9", then the AI ​​output results in the left table will select models in descending order of their blending ratios, with the cumulative blending ratio value being 0.9 or less. Since the blending ratios of Model A and Model B output by the AI ​​are 0.60 and 0.20 respectively, the cumulative ratio value becomes 0.80, and Model A and Model B are selected.

[0132] The mixing ratio values ​​output by the AI ​​can be used directly in the prediction calculation described later, or adjustments can be made so that the mixing ratio of Model A and Model B becomes 1. If adjustments are made, for example, one could adjust the values ​​so that the sum of the mixing ratios of Model A and Model B becomes 1 without changing the ratio itself. If adjustments are made in this way, the adjusted mixing ratio values ​​of Model A and Model B will be 0.75 and 0.25, respectively.

[0133] Equation (1) shows how to calculate the future predicted value in the example described above. When the AI-DFC model mixing ratio data 800 outputs a mixing ratio of 0.75 for model A and a mixing ratio of 0.25 for model B, the predicted value is calculated as shown in Equation (1) using the predicted values ​​for model A and model B. If the predicted values ​​for model A and model B are a and b, respectively, the final predicted value is calculated as 0.75a + 0.25b.

[0134] The future prediction value is the sum of the values ​​obtained by multiplying the prediction value using the selected model by the mixing ratio of that model, and then adding these values ​​together for each selected model. In other words, it is a process of summing the prediction values ​​from each selected prediction model according to the weight ratio of the prediction models.

[0135] In other words, this is an example of a process that obtains the weights of multiple acquired prediction models, and then calculates future predicted values ​​for the target of prediction based on the predicted values ​​calculated using each acquired prediction model and the weights associated with that prediction model.

[0136] Furthermore, this is an example of a process that calculates predicted values ​​by summing the predicted values ​​from each of the prediction models selected for calculating the predicted values, according to the weight ratios of the prediction models.

[0137] In this embodiment, the mixing ratios are output so that the sum is 1, but percentages may be used so that the sum is 100, or the maximum or minimum mixing ratio value X may be used as a reference, such as 0.1X or 2X. In other words, any method of expression is acceptable as long as it represents the likelihood of a candidate model or the probability that it could be a candidate.

[0138] By displaying the mixing ratio for each candidate model, users can learn the basis for which model was deemed more likely and used to calculate the predicted values.

[0139] Furthermore, by evaluating the likelihood (probability of selection) of each model as a mixing ratio along with model candidates from the trained model, and using this value to calculate future predictions, the more likely model will have a greater impact on the prediction, enabling accurate predictions using multiple models.

[0140] The first embodiment has been described above. If you want to improve the accuracy of AI-based predictions, you can try changing the attribute information and demand characteristics provided to the AI, as well as the method for selecting the prediction model described above. For example, if models A and B are frequently selected for a particular product, and selecting model C would improve prediction accuracy, then resetting the selection method to include model C would improve accuracy. In other words, by providing various methods for selecting prediction models, it becomes possible to efficiently adjust and set the model selection method that is appropriate for each product.

[0141] Furthermore, as mentioned in Figure 6, the item site master 600 allows for setting whether to use a model selected using AI output or a model specified by the user for each product or SKU. Here, in addition to indicating whether the prediction model was selected using AI, it may also be possible to display information identifying the person who specified the prediction method.

[0142] Specifically, if the system is configured to use a user-specified model for forecasting, the username and attributes of the user who set up that model will be displayed. For example, user attributes may include whether the user is a product manager at the company providing the demand forecasting system, or a customer using the demand forecasting system. Visualizing user attributes makes it possible to identify whether the model was specified by a product manager or a customer, which can be used as a reference when adjusting the model selection method. For example, when a customer adjusts the model selection, they can use the model set up by the product manager as a reference for verification.

[0143] Furthermore, when selecting a model using AI, information identifying the person who trained or retrained the AI ​​may be displayed. For example, in this embodiment, training data 1000 is created, and the username and user attributes of the person who provided the data to the model are displayed. This makes it possible for customers to distinguish and verify new trained models from those created by product managers when creating new trained models themselves.

[0144] Furthermore, in product demand forecasting, the forecast results may be displayed based on whether or not the forecasting model was selected using AI. For example, on a screen that displays forecast results in a time series, AI-based forecast results could be displayed in a different color. It is sufficient for the user to be able to identify which selection method was used to produce the forecast, and methods such as displaying a pop-up may also be used.

[0145] The above is an example of a process that clearly displays whether the future predicted value for the target of prediction is a predicted value obtained using a trained model or a predicted value based on a prediction model specified by the user.

[0146] (Second Embodiment) In the first embodiment, a method for calculating predicted values ​​from the output results of a trained model was shown. In the second embodiment, a method for identifying the prediction model to be used to calculate the prediction result based on the calculated prediction error and calculating future predicted values ​​is shown.

[0147] In steps S909 to S912 of Figure 9, forecast values ​​are calculated for the set forecast model candidates using past demand data, and the forecast error between these forecast values ​​and the forecast values ​​during the evaluation period is calculated to select the optimal model. In the second embodiment, in the target cycle of the product to be forecasted, the forecast model to be used to calculate forecast values ​​is selected using the forecast values ​​and forecast error of the forecast model calculated in this optimal time series forecast model selection process, and future forecast values ​​are calculated.

[0148] Figure 14 shows an example of a demand characteristic classification master 1400 in an embodiment of the present invention. The demand characteristic classification master 1400 includes, for example, items such as demand characteristic classification ID, demand characteristic classification name, DFC forecast category, forecast error, and error margin rate.

[0149] The demand characteristic classification ID is an arbitrary ID associated with the demand characteristic classification name. The demand characteristic classification name is an item that classifies demand characteristics according to the characteristics of the product. For example, it may include classifications such as seasonal products or products that are out of stock intermittently. The demand characteristic classification can be automatically assigned based on past demand performance, or it can be manually set by the user for each item site or product.

[0150] In the DFC prediction category, you can select either "DFC Selection" or "DFC Selection Mix Calculation". If DFC Selection is selected, the prediction value is calculated using the prediction model with the smallest prediction error (optimal time series prediction model) selected from the prediction error results calculated in the same way as in step S912, and this value is used as the future prediction value.

[0151] If DFC selection mix calculation is selected, the prediction error of the candidate models is calculated, a model is selected based on the prediction error, and future predictions are calculated based on the predicted values ​​of the selected model and their prediction errors. Details will be described later using Figure 12.

[0152] In the "Prediction Error" section, you can specify evaluation metrics used to assess the prediction model. For example, you can set the Mean Squared Error (RMSE) or the Mean Absolute Error (MAE). Using the evaluation metrics set here, the error between the predicted value and past demand performance is evaluated, and the prediction model selection process is performed.

[0153] The error range ratio is an item that accepts the method for selecting models to calculate predicted values ​​based on the prediction errors of candidate models when DFC selection mix calculation is set. As will be explained in detail in Figure 12, by using the error range ratio as the selection method, it is possible to select models whose errors fall within a range based on the prediction error value of the model with the smallest prediction error. This makes it possible to select models with more accuracy than simply selecting a specified number of models with small errors.

[0154] In other words, this is an example of a process in which a prediction model that satisfies predetermined conditions based on the error of the prediction model with the smallest error is selected as the prediction model to be used to calculate the predicted value.

[0155] Figure 12(2) shows the method for calculating predicted values ​​using the results of the time series forecasting model calculation process. In other words, it is a method for calculating predicted values ​​using the results of the forecasting error calculation of the forecasting model performed in steps S909 to S912 of Figure 9.

[0156] Figure 12(2) shows an example of the processing when "DFC Selection Mix Calculation" is set in the "DFC Prediction Classification" item of Figure 14. This shows an example of how to calculate the prediction error from the predicted values ​​of the model candidates in the prediction error calculation process in step S910, and select the model to be used in the prediction result calculation process based on the selection method set in the demand characteristic classification master 1400.

[0157] The forecast error output, as shown in the table on the left, displays the forecast error value calculated using actual demand data for each candidate model.

[0158] Based on the output results of the prediction error, a model to be used to calculate future predicted values ​​is selected based on the selection method set in the demand characteristic classification master 1400, and the predicted values ​​are calculated. In other words, this is an example of a calculation process in which the error between the predicted values ​​based on the prediction model and the time series data is calculated using the time series data related to the prediction target and the prediction model for calculating future predicted values, and based on this error, multiple prediction models are selected, and the future predicted values ​​related to the prediction target are calculated using the multiple prediction models.

[0159] For example, if the "Error Range" is set to "0.4" in the demand characteristic classification master 1400, then both Model A, which has the smallest prediction error in the output results of the prediction error in the left table, and Model B, which has an error of 40% or less, will be selected. In other words, this is an example of a process that selects multiple prediction models to be used to calculate the prediction value by the calculation means, based on conditions related to the magnitude of the error value of the prediction model.

[0160] The prediction errors for Model A and Model B are 130 and 180, respectively. The reciprocal of these values ​​is used to calculate the mixing ratio of Model A and Model B. A smaller prediction error indicates better prediction accuracy, so the reciprocal of the prediction error is used as the mixing ratio. Alternatively, the mixing ratio can be adjusted so that the sum of the reciprocals of the prediction errors of Model A and Model B equals 1, without changing the ratio of the reciprocals of the prediction errors of Model A and Model B. If this adjustment is made, the adjusted mixing ratios of Model A and Model B will be 0.58 and 0.42, respectively.

[0161] Equation (2) shows how to calculate future predicted values ​​in the example described above. When the mixing ratio in Model A is output as 0.58 and the mixing ratio in Model B is output as 0.42, the predicted value is calculated as shown in Equation (2) using the predicted values ​​obtained using Model A and Model B. If the predicted values ​​obtained using Model A and Model B are a and b, respectively, the final predicted value is calculated as = 0.58a + 0.42b. In other words, this is an example of a process that calculates future predicted values ​​for the target of prediction based on predicted values ​​obtained using multiple prediction models and the error value of the prediction model.

[0162] Furthermore, this is an example of a process that accepts a method for selecting a prediction model from among the prediction models for which errors have been calculated, and then calculates the prediction value using the prediction model specified by the accepted selection method.

[0163] In other words, the future predicted value is the sum of the values ​​obtained by multiplying the predicted value using the selected model by the mixing ratio of that model, and then adding these values ​​together for each selected model. That is, it is a process of summing the predicted values ​​from each selected prediction model according to the ratio of the weights of the prediction models. In other words, it is an example of a process that calculates the predicted value by summing the predicted values ​​from each prediction model selected to be used in calculating the predicted value, according to the ratio of the reciprocals of the errors of the prediction models.

[0164] In the second embodiment, a method for selecting a model to be used to calculate future predicted values ​​from among the model candidates by setting an error margin ratio was described, but other selection methods may also be used.

[0165] For example, similar to the selection method in the "prediction model selection category" of the first embodiment, an upper limit on the number of prediction models may be set as the top specified number, and the specified number of models may be selected in order of smallest error. In other words, this is an example of a process that selects a predetermined number of prediction models in order from the prediction model with the smallest error.

[0166] Alternatively, you can set a value for the cumulative prediction error and select models such that the cumulative prediction error value is less than or equal to a specified value. Or, you can set a value for the prediction error and select models such that the prediction error value is less than or equal to a specified value. In other words, this is an example of a process that selects prediction models in order from those with small errors until the sum of the prediction errors satisfies a predetermined condition.

[0167] This allows users to configure how future predictions are calculated and how to select the models used to calculate them. By calculating the mixing ratio based on the magnitude of the error, models with small errors will have a greater impact on the predictions, while models with large errors will have a smaller impact, enabling accurate predictions using multiple models.

[0168] Although embodiments have been described above, the present invention can take the form of, for example, a system, apparatus, method, program, or recording medium. Specifically, it may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device.

[0169] Furthermore, the program in this invention is a program that allows a computer to execute the processing method shown in the flowchart, and the storage medium of this invention stores a program that allows a computer to execute the processing method shown in the flowchart. Note that the program in this invention may also be a program for each processing method of each device shown in the diagram.

[0170] As described above, it goes without saying that the object of the present invention can also be achieved by supplying a recording medium containing a program that realizes the functions of the embodiments described above to a system or device, and by having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0171] In this case, the program read from the recording medium itself realizes the novel function of the present invention, and the recording medium on which that program is recorded constitutes the present invention.

[0172] For recording media used to supply programs, examples include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, and the like.

[0173] Furthermore, it goes without saying that the functions of the aforementioned embodiments are realized not only by the computer executing the program it has read, but also by the operating system (OS) running on the computer performing some or all of the actual processing based on the instructions of that program, thereby realizing the functions of the aforementioned embodiments.

[0174] Furthermore, it goes without saying that this also includes cases where, after a program read from a recording medium is written to the memory of a function expansion board inserted into a computer or a function expansion unit connected to a computer, the CPU or other components of the function expansion board or function expansion unit perform some or all of the actual processing based on the instructions of the program code, and the functions of the aforementioned embodiments are realized through that processing.

[0175] Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device. It goes without saying that the present invention can also be applied when the results are achieved by supplying a program to a system or device. In this case, by reading a recording medium containing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

[0176] Furthermore, by downloading and reading the program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that configurations combining the above-described embodiments and their variations are all included in the present invention. [Explanation of Symbols]

[0177] 100 Supply and Demand Management System 110 Client terminals 120 Networks 130 servers

Claims

1. A selection means that uses time series data related to the target of prediction and a prediction model for calculating future predicted values ​​to calculate the error between the predicted values ​​based on the prediction model and the time series data, and selects a plurality of prediction models based on the error, A calculation means for calculating future predicted values ​​related to the target of prediction using the aforementioned multiple prediction models, An information processing device characterized by comprising:

2. The information processing apparatus according to claim 1, characterized in that the calculation means calculates a future predicted value relating to the prediction target based on the predicted value calculated using the plurality of prediction models selected by the selection means and a value relating to the error of the prediction model.

3. The information processing apparatus according to claim 2, characterized in that the value relating to the error is a value based on the reciprocal of the error.

4. The system includes a receiving means for receiving a method for selecting a prediction model from among the prediction models whose errors have been calculated by the selection means, to be used for calculating the predicted value by the calculation means, The information processing apparatus according to claim 1, characterized in that the calculation means calculates a predicted value using a prediction model specified by the selection method received by the reception means.

5. The information processing apparatus according to claim 4, characterized in that the selection method is a method for selecting the plurality of prediction models to be used for calculating the predicted value by the calculation means based on conditions based on the magnitude of the error value of the prediction model.

6. The information processing apparatus according to claim 4, characterized in that the selection method is a method of selecting from the prediction models in order from the prediction model with the smallest error until the sum of the prediction errors satisfies a predetermined condition.

7. The information processing apparatus according to claim 4, characterized in that the selection method is a method of selecting a prediction model that satisfies predetermined conditions based on the error of the prediction model with the smallest error among the prediction models, as the prediction model to be used for calculating the prediction value by the calculation means.

8. The information processing apparatus according to claim 4, characterized in that the selection method is a method of selecting a predetermined number of prediction models from among the prediction models in order from the prediction model with the smallest error.

9. The information processing apparatus according to claim 4, characterized in that the calculation means calculates the predicted value by summing the predicted values ​​from each of the prediction models selected as prediction models used to calculate the predicted value, in accordance with the ratio of the reciprocals of the errors between the prediction models.

10. The information processing apparatus according to claim 1, characterized in that the selection means selects a plurality of prediction models from a set of prediction models to calculate future predicted values ​​related to the prediction target based on the error.

11. The information processing apparatus according to claim 1, characterized in that the calculation means calculates the future demand for the product related to the forecast target using a plurality of forecast models selected by the selection means.

12. The selection means of the information processing device includes a selection step in which it uses time-series data relating to the target of prediction and a prediction model for calculating future predicted values ​​to calculate the error between the predicted values ​​based on the prediction model and the time-series data, and selects a plurality of prediction models based on the error, The calculation means of the information processing device calculates future predicted values ​​related to the prediction target using the plurality of prediction models, An information processing device characterized by comprising:

13. A program for causing at least one computer to function as one of the means of an information processing apparatus described in any one of claims 1 to 11.

Images