Prediction device, prediction method, and prediction program

Abstract

To provide a technique capable of performing ensemble prediction with high accuracy even when a distribution of information related to a prediction target locally changes.SOLUTION: The prediction device is configured to, based on an evaluation result obtained by evaluating performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and a true value corresponding to the model input information, and the evaluation information, A weight update unit that updates some or all of the plurality of first weight vectors and some or all of the plurality of second weight vectors, and an integration unit that integrates a prediction result predicted by each model with reference to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors on the basis of the prediction target information, and SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to techniques for making predictions. [Background technology]

[0002] There are known techniques for dynamically changing the weights assigned to each model in ensemble prediction. For example, Patent Literature 1 describes a technique for calculating the weights assigned to each model depending on the degree of agreement between the predicted values ​​predicted by each model based on detected values ​​obtained during a predetermined retroactive period from the time of the prediction target and the actual values ​​obtained during that period. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2016-45799 Summary of the Invention [Problem to be solved by the invention]

[0004] The technique described in Patent Document 1 has a problem in that the accuracy of ensemble prediction decreases when the distribution of information related to the prediction target changes locally.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and one exemplary purpose thereof is to provide a technology for performing ensemble prediction with high accuracy even when the distribution of information related to the prediction target changes locally. [Means for solving the problem]

[0006] A prediction device according to an exemplary aspect of the present disclosure includes a weight update means for updating some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; and a prediction means for integrating prediction results predicted by each model by referring to model input information included in prediction target information related to the prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputting an integrated prediction result.

[0007] A prediction method according to an exemplary aspect of the present disclosure includes: a weight update process in which at least one processor evaluates the performance of each of a plurality of models by referring to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and updates some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on the evaluation information; and a prediction process in which the at least one processor integrates the prediction results predicted by each model by referring to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputs an integrated prediction result.

[0008] A prediction method according to an exemplary aspect of the present disclosure is a program that causes a computer to function as a prediction device, and causes the computer to function as: a weight update means that updates some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on evaluation results obtained by evaluating the performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information; and a prediction means that outputs an integrated prediction result obtained by integrating the prediction results predicted by each model with reference to model input information included in prediction target information related to the prediction target, using the plurality of first weight vectors and a weight vector selected from the plurality of second weight vectors based on the prediction target information. [Effects of the Invention]

[0009] According to an exemplary aspect of the present disclosure, an exemplary effect is achieved in that a technology can be provided that performs ensemble prediction with high accuracy even when the distribution of information related to the prediction target changes locally. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a prediction device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram showing the flow of a prediction method according to the present disclosure. [Figure 3] FIG. 1 is a block diagram illustrating a configuration of a prediction device according to the present disclosure. [Figure 4] FIG. 10 is a diagram for explaining processing by a prediction device according to the present disclosure. [Figure 5] FIG. 10 is a flow diagram illustrating an example of a detailed flow of a weight update process according to the present disclosure. [Figure 6] FIG. 10 is a flow diagram illustrating an example of a detailed flow of a prediction process according to the present disclosure. [Figure 7] FIG. 1 is a block diagram illustrating a configuration of a prediction device according to the present disclosure. [Figure 8] FIG. 10 is a diagram illustrating a display example by the prediction device according to the present disclosure. [Figure 9] FIG. 1 is a schematic diagram illustrating a specific example of a prediction device applied in the medical field according to the present disclosure. [Figure 10] FIG. 1 is a schematic diagram illustrating a specific example of a prediction device applied in the retail field according to the present disclosure. [Figure 11] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a computer that functions as each device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] First Exemplary Embodiment A first exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referred to in describing this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise.

[0013] (Configuration of prediction device) The configuration of the prediction device 1 will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of the prediction device 1. As shown in FIG. 1, the prediction device 1 includes a weight update unit 11 and a prediction unit 12. The weight update unit 11 updates some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on evaluation results obtained by evaluating the performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information. Here, the model input information for evaluation is, for example, information obtained over time after the start of operation of the plurality of models. The second weight vector is a vector whose components are weights assigned to each of the plurality of first weight vectors. The weight update unit 11 also functions as an acquisition unit for acquiring the evaluation information.

[0014] The prediction unit 12 outputs an integrated prediction result obtained by integrating prediction results obtained by each model with reference to model input information included in prediction target information related to the prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information. As an example, the prediction unit 12 outputs an integrated prediction result obtained by integrating prediction results obtained by each model with a second weight vector selected based on the prediction target information and a first weight vector selected in accordance with the selected second weight vector. Note that, as an example, the second weight vector selected in accordance with the prediction target information is a single vector. Furthermore, the first weight vector selected in accordance with the selected second weight vector may be a single vector or multiple vectors. Note that the prediction unit 12 also functions as an acquisition unit for acquiring the prediction target information.

[0015] Furthermore, the prediction target is an object to be predicted using each model, and is, for example, sales amount, hospital bed occupancy rate, classification of human behavior, etc., but is not limited to these. The prediction target is also called, for example, a dependent variable. Furthermore, model input information is information input to each model, and is also called an explanatory variable. When the prediction target is sales amount on a target day, the model input information may include, for example, the weather on the target day. When the prediction target is a hospital bed occupancy rate one week from now, the model input information may include the most recent hospital bed occupancy rate. When the prediction target is a classification of human behavior, the model input information may include an image of the person.

[0016] The model input information for evaluation is information obtained over time after the start of operation and is different from the model input information used to evaluate the performance of each model when the model was generated. The model input information for evaluation may be, but is not limited to, model input information (e.g., the explanatory variables described above) included in the prediction target information referenced by the prediction unit 12 in past prediction processes. The evaluation information includes true values ​​corresponding to the model input information for evaluation, and the weight update unit 11 evaluates the performance of each model and the prediction performance of each first weight vector based on the true values. The weight update unit 11 selects some or all of the multiple first weight vectors as update targets based on the evaluation information, and updates the selected first weight vectors based on the evaluation results of the performance of each model. The weight update unit 11 updates some or all of the multiple second weight vectors based on the evaluation results of the first weight vectors. However, these examples do not limit the present exemplary embodiment.

[0017] (Effect of prediction device) As described above, the prediction device 1 is configured to include the weight update unit 11 and the prediction unit 12. Therefore, the prediction device 1 can accurately update some or all of the multiple first weight vectors and some or all of the multiple second weight vectors based on the evaluation results of each model evaluated with reference to the evaluation information, as well as the evaluation information referenced to obtain the evaluation results. Furthermore, when making predictions, weight vectors selected from the multiple updated first weight vectors and multiple second weight vectors based on the prediction target information are used. Therefore, even when the distribution of model input information included in the prediction target information changes locally, ensemble predictions can be made with high accuracy.

[0018] (Example of program implementation) When the prediction device 1 is configured by a computer including at least one processor and a memory, the memory stores the following prediction program. The prediction program causes a computer to function as the prediction device 1, and causes the computer to function as: a weight update unit 11 that updates some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on evaluation results obtained by evaluating the performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; and a prediction unit 12 that outputs an integrated prediction result obtained by integrating prediction results obtained by each model with reference to model input information included in prediction target information related to the prediction target, using the plurality of first weight vectors and a weight vector selected from the plurality of second weight vectors based on the prediction target information.

[0019] (Prediction method flow) The flow of prediction method S1 will be described with reference to FIG. 2. FIG. 2 is a flow diagram showing the flow of prediction method S1. As shown in FIG. 2, prediction method S1 includes weight update processing S11 and prediction processing S12. In weight update processing S11, at least one processor updates some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on evaluation results obtained by evaluating the performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information. Here, the model input information for evaluation is, for example, information obtained over time after the start of operation of the plurality of models. Furthermore, the second weight vector is a vector whose components are weights assigned to each of the plurality of first weight vectors.

[0020] Next, in prediction processing S12, at least one processor outputs an integrated prediction result by integrating the prediction results predicted by each model by referring to model input information included in prediction target information related to the prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information.

[0021] At least one processor may repeatedly execute the weight update process S11 and the prediction process S12. However, the weight update process S11 and the prediction process S12 may be executed independently of each other, and the execution order and timing of each process are not specified. For example, the prediction process S12 is not necessarily executed after the weight update process S11. The weight update process S11 may be executed repeatedly at any timing, and the prediction process S11 may be executed repeatedly in response to a prediction request.

[0022] (Effect of forecasting method) As described above, prediction method S1 employs a configuration including the weight update process S11 and prediction process S12. Therefore, prediction method S1 can accurately update some or all of the multiple first weight vectors and some or all of the multiple second weight vectors based on the evaluation results of each model evaluated with reference to the evaluation information, as well as the evaluation information referenced to obtain the evaluation results. Furthermore, when making predictions, weight vectors selected from the multiple updated first weight vectors and multiple second weight vectors based on the prediction target information are used. This provides the advantage of enabling highly accurate ensemble predictions even when the distribution of model input information included in the prediction target information changes locally.

[0023] Second Exemplary Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0024] (Configuration of prediction device) The configuration of the prediction device 1A will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the prediction device 1A. In addition to the weight update unit 11 and prediction unit 12 that the prediction device 1 includes, the prediction device 1A also includes a model storage unit 13 and a weight vector storage unit 14.

[0025] (Model storage section) The model storage unit 13 stores Nmodel models f_1, f_2, ..., f_Nmodel, where Nmodel is a natural number equal to or greater than 2. In other words, the model storage unit 13 stores a model set F expressed by the following formula (1). The model set F may also be referred to as a model pool MP.

number

[0026] As an example, a case will be described in which the prediction target is the sales amount for a target date. In this example, the sales forecast values ​​output by each model f_i with reference to the same model input information may differ from one another. For example, the model input information may include store surroundings images taken around the store on the target date, weekday / holiday labels indicating weekdays or holidays, and weather. Each model f_i may be a model that references the model input information and outputs the sales forecast value for the target date as the prediction result y_i.

[0027] Here, the prediction target information referred to by the prediction unit 12, the details of which will be described later, includes at least model input information. Furthermore, the prediction target information may further include, in addition to the model input information, additional information that is not input to each model f_i. Furthermore, the evaluation information referred to by the weight update unit 11, the details of which will be described later, includes at least model input information for evaluation. Furthermore, the evaluation information may further include additional information for evaluation in addition to the model input information for evaluation.

[0028] For example, when the model input information includes a store surrounding image, examples of the additional information include the resolution of the store surrounding image, the time of shooting, the type of shooting device, the photographer, or a combination of these. However, examples of the model input information and the additional information are not limited to these. Note that the prediction target information and the evaluation information only need to include at least the model input information, and do not necessarily need to include the additional information. However, when multiple conditions described below are determined with reference to the additional information, both the prediction target information and the evaluation information include the model input information and the additional information.

[0029] Each model f_i may be a machine learning model or a model that is not a machine learning model. For example, examples of machine learning models include, but are not limited to, deep neural networks (DNNs), gradient boosting decision trees (GBDTs), linear regression models, etc. Examples of models that are not machine learning models include, but are not limited to, rule-based models, etc. At least two models f_i1 and f_i2 (i1 ≠ i2) included in the model set F may be the same type of model or different types of models. Note that if at least two models f_i1 and f_i2 included in the model set F are the same type of machine learning models, the two models f_i1 and f_i2 may have been trained using at least partly different training datasets or may have different hyperparameters. Note that model f_i is referred to as f i The predicted result y_i is written as y i It is sometimes written as:

[0030] (Weight vector storage unit) The weight vector storage unit 14 includes: Nweight(1) first weight vectors w (1) _1, w (1) _2, ..., w (1) _Nweight(1) and Nweight(2) second weight vectors w (2) _1, w(2) _2, ..., w (2) _Nweight(2) and Here, Nweight(1) and Nweight(2) are, for example, natural numbers greater than or equal to 2. (1) _j, w (1) j Or write it as w (2) _k, w (2) k In other words, the weight vector storage unit 14 stores the first weight set W (1) and the second weight set W (2) I remember that.

number

number

[0031] (first weight vector) The first weight vector w (1) _j is the weight w of Nmodels (1) _j_i as element (w (1) ji , or w (1) j,i It is a vector whose σ is a vector of σ, and is expressed by the following equation (4).

number

[0032] In addition, the plurality of first weight vectors w stored in the weight vector storage unit 14 (1)Each of the first conditions c_j is a plurality of first conditions that can be satisfied by the prediction target information, and is associated with at least one of a plurality of first conditions that can be satisfied by the evaluation information. (1) There may be multiple first conditions c (1) Each of the first conditions c_j may be a condition that the model input information can satisfy. For example, if the model input information includes a weekday / holiday label and a weather label, the first conditions c_j may be a condition that the model input information can satisfy. (1) _1, c (1) _2, condition c (1) _1 "Sunny weekday" and condition c (1) In addition, when the evaluation information and the prediction target information include additional information, a plurality of first conditions c (1) Each of the j's may be a condition that the additional information may satisfy, or may be a condition that both the model input information and the additional information may satisfy.

[0033] Here, the first weight vector and the first condition do not necessarily correspond one-to-one. However, the following description focuses on an example in which there is a one-to-one correspondence, and the first weight vector w (1) The first condition that corresponds one-to-one with _j is the first condition c (1) In other words, the first weight vector w (1) _j is the first condition c (1) The first weight vector w (1) _j and the first condition c (1) If there is a one-to-one correspondence between _j and multiple first conditions c (1) The number of _j is equal to the number of the first weight vectors Nweight(1). Note that the first condition c (1) _j is c (1) j It can also be written as:

[0034] (second weight vector) On the other hand, the second weight vector w (2) _k is Nweight(1) weights w (2) _k_j as element (w(2) kj , or w (2) k,j It is a vector whose σ is a vector of σ, and is expressed by the following equation (5).

number

[0035] In addition, the plurality of second weight vectors w stored in the weight vector storage unit 14 (2) Each of the second conditions c_k is a plurality of second conditions that can be satisfied by the prediction target information, and is associated with at least one of the plurality of second conditions that can be satisfied by the evaluation information. (2) It is preferable that the number of second conditions c_k is set to one. (2) Each of the second conditions c_k may be a condition that the model input information can satisfy. (2) _k satisfies the first condition c (1) For example, if the model input information includes a weekday / holiday label and a weather label, multiple first conditions c (2) _1, c (2) _2, condition c (2) _1 "Sunny weekday" and condition c (2) In addition, when additional information is included in the evaluation information and the prediction target information, a plurality of second conditions c (2) Each of the .sub.k may be a condition that the additional information may satisfy, or may be a condition that both the model input information and the additional information may satisfy.

[0036] Here, the second weight vector and the second condition do not necessarily correspond one-to-one. However, the following description focuses on an example in which there is a one-to-one correspondence, and the second weight vector w (2) The second condition that corresponds one-to-one with _k is the second condition c (2) In other words, the second weight vector w (2) _k is the second condition c (2) The second weight vector w (2) _k and the second condition c (2) If there is a one-to-one correspondence between _k and multiple second conditions c (2) The number of _k is equal to the number of second weight vectors, Nweight(2). Note that the second condition c (2) _k is c (2) k It can also be written as:

[0037] The second weight vector w (2) _k is the first weight vector w as described above. (1) The weight assigned to _j is the component w (2) _k_j. Therefore, the second weight vector w (2) _k is a weight vector used in the prediction process. (1) It can also be expressed as a weight vector for softly determining which of the first weight vectors w_j is to be used. Here, "softly determining" means, for example, (1) This refers to the use of a combination of _j using multi-stage coefficients.

[0038] Figure 4 shows First weight vector w (1) j (j=1, ,7) First condition c (1) j (j=1, ,7) The second weight vector w (2) k (k=1, ,4) Second condition c (2) k(k=1, ,4) As shown in FIG. 4, the first weight vector w (1) j Each of (j=1, , 7) has the first condition c (1) j (j=1,...,7) is associated with the second weight vector w (2) k Each of (k=1, , 4) has the second condition c (2) k (k=1,···,4) are associated with each other.

[0039] Also, as shown in Figure 4, the second weight vector w (2) k is the first weight vector w (1) j More specifically, in FIG. 4, the second weight vector w (2) 1 is the first weight vector w (1) j Seven components (elements) corresponding to the number of w (2) 1=(w (2) 11 ,w (2) 12 ,w (2) 13 ,w (2) 14 ,w (2) 15 ,w (2) 16 ,w (2) 17 ) Among these, ·w (1) Component w corresponding to 1 (2) 11 ·w (1) Component w corresponding to 2 (2) 12 ·w (1) Component w corresponding to 5 (2) 15 This shows the case where has a large weight (corresponding to the arrow in Figure 4).

[0040] In this exemplary embodiment, the plurality of first weight vectors w (1) _j has multiple first conditions c (1) 4, the weight vector may be configured to include a weight vector associated with a condition obtained by combining two or more conditions included in the first condition c (1) j (j=1, ,7) has First condition c (1) 1. "Sunny weekday" and the first condition c (1) 2. Condition c obtained by integrating with "sunny holiday" (1) 5. Sunny Similarly, the plurality of first conditions c (1) j (j=1, ,7) has First condition c (1) 3. "Rainy weekday" and the first condition c (1) 4. Condition c obtained by integrating with "rainy holiday" (1) 6. Rain Furthermore, the plurality of first conditions c (1) j (j=1, ,7) has First condition c (1) 5 "Sunny" and the first condition c (1) 6. Condition c obtained by integrating with "rain" (1) 7 "All Samples" Contains:

[0041] The first condition obtained by this integration (c (1) 5,c (1) 6,c (1) 7) and a first weight vector (w (1) 5,w (1) 6,w (1) 7) is set. Furthermore, such a first weight vector (w (1) 5,w (1) 6,w (1) 7) weight (w (2) k5 ,w (2)k6 ,w (2) k7 ) as a component of the second weight vector w (2) k By adopting such a configuration, even if the first condition c (1) j Even if the settings are not necessarily appropriate, a suitable ensemble prediction can be performed.

[0042] A condition obtained by integrating multiple conditions can also be expressed as, for example, a condition that expands multiple conditions, a condition that reflects the characteristics of multiple conditions, or a condition that is derived from multiple conditions. C1:1≦x<100 C2: 100≦x<200 condition C3, which is obtained by combining conditions C1 and C2, is the condition obtained by the subspace of C1 and the subspace of C2. C3: 50≦x<150 or a condition extended to include both condition C1 and condition C2. C3: - 50≦x<250 Furthermore, a condition obtained by combining a plurality of conditions may be a condition obtained by the logical sum of the plurality of conditions. The same applies to other parts of this specification.

[0043] (Prediction Department) The prediction unit 12 predicts a plurality of second conditions c (2) _k, the second condition c that the prediction target information satisfies (2) The second weight vector w associated with k (2) The prediction unit 12 also selects prediction results y_i output from each of the multiple models f_i for model input information included in the prediction target information, as follows: The selected second weight vector w (2) _k, and The selected second weight vector w (2) The first weight vector w is indirectly selected by k (1)_j combination For example, in the case of a regression task, the process of calculating the integrated prediction result is expressed by the following equation (6).

number

[0044] For classification tasks, for example, i (x) represents a vector of the number of classes, and f i The predicted probability for each class label i is expressed by (x). The prediction unit 12 can then calculate the integrated predicted probability using an equation similar to the above equation (6). When finally determining a label as a predicted value, the prediction unit 12 determines the class label with the highest probability as the label as the predicted value.

[0045] Furthermore, when the prediction target information includes additional information, the prediction unit 12 calculates a plurality of second weight vectors w based on the model input information and the additional information included in the prediction target information. (2) k In this case, w in equation (6) (2) k,j is the second condition c that is satisfied by either or both of the model input information and the additional information. (2) k The second weight vector w associated with (2) k The first weight vector w (1) j It may also be an element associated with

[0046] For example, an example will be described in which the prediction target information includes store surroundings images and weekday / holiday labels as model input information, and the resolution of the store surroundings images as additional information. In this example, the second condition c (2) 1 is "weekday and high resolution", and the second condition c (2) 2 is "weekday and low resolution", and the second condition c (2) 3 is "holiday and high resolution", and the second condition c (2) 4 may be “holiday and low resolution.” In this case, the number of second weight vectors Nweight(2) may be 4, which is the number of conditions.

[0047] Furthermore, for example, a set of prediction target information may be input to the prediction unit 12. Such a set X is expressed by the following equation (7).

number

[0048] In this case, the prediction unit 12 outputs a set Y of integrated prediction results corresponding to the set X. Such a set Y is expressed by the following equation (8).

number

[0049] (Weight update section) The weight update unit 11 is configured in the same manner as in the first exemplary embodiment, and is further configured as follows: The weight update unit 11 is configured to update the weights of a first plurality of conditions c (1) _j, the first condition c that the evaluation information satisfies (1) The first weight vector w associated with _j (1) For example, if the model input information for evaluation included in the evaluation information includes a weekday / holiday label indicating a weekday and a weather label indicating fine weather, the evaluation information satisfies the first condition c (1) Therefore, the weight update unit 11 satisfies the first condition c (1) The first weight vector w associated with _1 (1) _1 is the item to be updated.

[0050] The weight update unit 11 also updates the weights based on a second plurality of conditions c (2) _k, the second condition c that the evaluation information satisfies (2) The second weight vector w associated with k (2) For example, if the model input information for evaluation included in the evaluation information includes a weekday / holiday label indicating a weekday and a weather label indicating fine weather, the evaluation information satisfies the second condition c (2) Therefore, the weight update unit 11 satisfies the second condition c (2) The second weight vector w associated with _1 (2) _1 is the item to be updated.

[0051] Here, a case will be described in which model input information included in prediction target information referenced by the prediction unit 12 in a past prediction process is applied as the model input information for evaluation to be included in the evaluation information. For example, the weight update unit 11 may use model input information referenced by the prediction unit 12 in the past (e.g., store surroundings image and weekday / holiday label for a target day) as model input information for evaluation in response to obtaining a true value (e.g., actual sales value for a target day) corresponding to the model input information. In this case, the weight update unit 11 may acquire the evaluation information including the model input information for evaluation and the true value. However, the model input information for evaluation to be included in the evaluation information is not limited to the above-mentioned example as long as it is information corresponding to the elapsed time since the start of operation of the multiple models f_i.

[0052] Furthermore, the weight update unit 11 updates the first plurality of weight vectors w based on the plurality of pieces of evaluation information and the evaluation results of the performance of each model f_i with reference to the plurality of pieces of evaluation information. (1) some or all of the j, and a second plurality of weight vectors w (2) The evaluation result of the performance of each model f_i with reference to a plurality of pieces of evaluation information may be, for example, a statistical value (e.g., average value, maximum value, minimum value, etc.) of the evaluation result of the performance of each model f_i with reference to each piece of evaluation information.

[0053] Furthermore, when the evaluation information includes additional information for evaluation, the weight update unit 11 updates the first plurality of weight vectors w based on the evaluation results of the performance of each model f_i with reference to the evaluation information, and the model input information for evaluation and the additional information for evaluation included in the evaluation information. (1) some or all of the j, and a second plurality of weight vectors w (2) Some or all of _k may be updated.

[0054] For example, the set of evaluation information Deval input to the weight update unit 11 is expressed by the following equation (9).

number

[0055] For example, the weight update unit 11 may update the weights of a plurality of first conditions c (1) For at least one of the first conditions c, the first condition c is selected from the plurality of pieces of evaluation information. (1) One or more pieces of evaluation information that satisfy the first condition c may be extracted. (1) The process of extracting one or more pieces of evaluation information that satisfy j is expressed by, for example, the following equation (10).

number

[0056] Furthermore, the weight update unit 11 evaluates the performance of each model f_i using the extracted one or more pieces of evaluation information (D(1)eval_j), and based on the evaluation result, updates the first condition c (1) The first weight vector w associated with _j (1) Update _j. Equation (10) shows the case where the evaluation information includes additional evaluation information, and c (1) _j(x,v) is the model input information x for evaluation and the additional information v for evaluation contained in the evaluation information, and the first condition c (1) If _j is satisfied, it is true, and if not, it is false.

[0057] For example, suppose that the set Deval contains five pieces of evaluation information, three of which contain the weekday / holiday label "weekday" and the remaining two contain the weekday / holiday label "holiday." In this case, the first condition c(1) The first three pieces of evaluation information that satisfy the first condition c are extracted as a subset D(1)eval_1. (1) The latter two pieces of evaluation information that satisfy _2 "holiday" are extracted as the subset D(1)eval_2.

[0058] Similarly, the weight update unit 11 updates a plurality of second conditions c (2) For at least one of the second conditions c_k, the second condition c is selected from the plurality of pieces of evaluation information. (2) One or more pieces of evaluation information that satisfy the second condition c may be extracted. (2) The process of extracting one or more pieces of evaluation information that satisfy _k is expressed by, for example, the following equation (11).

number

[0059] Furthermore, the weight update unit 11 evaluates the first weight vector using the extracted one or more pieces of evaluation information (D(2)eval_k), and based on the evaluation result, updates the second condition c (2) The second weight vector w associated with k (2) Equation (11) shows the case where the evaluation information includes additional evaluation information, and c (2) _k(x,v) is the model input information x for evaluation and the additional information v for evaluation contained in the evaluation information, and the second condition c (2) If _k is satisfied, it is true, and if not, it is false.

[0060] For example, suppose that the set Deval contains five pieces of evaluation information, three of which contain the weekday / holiday label "weekday" and the remaining two contain the weekday / holiday label "holiday." In this case, the second condition c (2)The first three pieces of evaluation information that satisfy the condition "weekday" are extracted as the subset D(2)eval_1. (2) The latter two pieces of evaluation information that satisfy _2 "holiday" are extracted as a subset D(2)eval_2.

[0061] (Prediction method flow) The prediction method S1A executed by the prediction device 1A configured as above will be described in substantially the same manner as the prediction method S1 described with reference to Fig. 2. However, the weight update process S11 and the prediction process S12 will be described in more detail below.

[0062] (Detailed flow of weight update process) First, a detailed flow of the weight update process S11 will be described with reference to Fig. 5. Fig. 5 is a flow diagram illustrating an example of a detailed flow of the weight update process S11. As shown in Fig. 5, the weight update process S11 includes steps S111 to S116.

[0063] (Step S111) In step S111, the weight update unit 11 acquires evaluation information. For example, when the prediction method S1A has been executed in the past, the weight update unit 11 may acquire evaluation information including, as model input information for evaluation, model input information included in the prediction target information used in the past prediction process S12. Furthermore, when the number of unprocessed evaluation information pieces among the acquired evaluation information pieces reaches a predetermined number, the weight update unit 11 may execute the processes from step S112 onwards using the predetermined number of evaluation information pieces.

[0064] (Step S112) Subsequently, in step S112, the weight update unit 11 selects, from the plurality of first conditions, the first condition c that is satisfied by the evaluation information acquired in step S111. (1) j As an example, the weight update unit 11 extracts a plurality of first conditions c (1) j (For example, among the first conditions shown in FIG. 4,(1) 1 "Sunny Weekday", c (1) 5 "Sunny", c (1) 7 "all samples", etc.) are extracted from the evaluation information set Deval. (1) j Here, an example of the evaluation information subset D(1)eval_j is as described with reference to equation (10).

[0065] (Step S113) Subsequently, in step S113, the weight update unit 11 updates the first condition c extracted in step S112. (1) j The first weight vector w corresponding to (1) j As an example, the weight update unit updates a plurality of first conditions c (1) 1 "Sunny Weekday", c (1) 5 "Sunny", c (1) 7. Multiple first weight vectors w corresponding to "all samples" (1) 1. W (1) 5. W (1) 7 is updated. For example, the evaluation information subsets D(1)eval_1, D(1)eval_5, and D(1)eval_7 extracted in step S112 are used in the update process of these first weight vectors. However, this is not intended to limit the present example.

[0066] More specifically, the weight update unit 11 ·Predicted value y^ of each model i =f i The evaluation results for each model are derived by referring to (x)(i=1,...,Nmodel) and the true value (correct value) y. The first weight vector w is calculated based on the derived evaluation result. (1) j element of the model f i The element w corresponding to (1) j,i Update Here, the specific update algorithm does not limit the present exemplary embodiment, but as an example, the Hedge algorithm

number

number

[0067] (Step S114) Subsequently, in step S114, the weight update unit 11 selects, from the plurality of second conditions, the second condition c that is satisfied by the evaluation information acquired in step S111. (2) k As an example, the weight update unit 11 extracts one second condition c that the evaluation information satisfies. (2) k (For example, the second condition c shown in Figure 4 (2) 1) is extracted from the evaluation information set Deval. (2) jHere, an example of the evaluation information subset D(2)eval_k is as described with reference to equation (11).

[0068] (Step S115) Subsequently, in step S115, the weight update unit 11 updates the second condition c extracted in step S114. (2) k The second weight vector w corresponding to (2) k As an example, the weight update unit updates one second condition c (1) 1. One second weight vector w corresponding to "sunny weekday" (2) 1 is updated. In the update process of the second weight vector, the subset D(2)eval_1 of the evaluation information extracted in step S114 is used, as an example, although this example is not limited to this.

[0069] More specifically, the weight update unit 11 updates the first weight vector w (1) j Using the above, the predicted value for the evaluation information is calculated based on the first condition c (1) j (j=1, . . . , Nweight(1)). In other words, the weight update unit 11 calculates

number

[0070] (Step S116) In step S116, the weight update unit 11 determines whether or not there is other evaluation information that has not yet been processed. If there is other evaluation information that has not yet been processed (YES in step S116), the processing from step S112 onwards is repeated. If not (NO in step S116), the processing ends.

[0071] (Detailed flow of prediction process) Next, details of the prediction process S12 will be described with reference to Fig. 6. Fig. 6 is a flow diagram showing an example of a detailed flow of the prediction process S12. As shown in Fig. 6, the prediction process S12 includes steps S121 to S126.

[0072] (Step S121) First, in step S121, the prediction unit 12 acquires a set X of prediction target information. The prediction unit 12 may acquire the set X of prediction target information stored in a memory included in the prediction device 1A, or may acquire the set X of prediction target information received via a network. Furthermore, the set X of prediction target information may include at least one piece of prediction target information, and is not limited to including a plurality of pieces of prediction target information. An example of the set X has been described with reference to equation (7) etc.

[0073] (Step S122) Subsequently, in step S122, the prediction unit 12 selects a second condition c that is satisfied by the prediction target information from the plurality of second conditions. (2) As an example, the prediction unit 12 extracts one second condition c_k determined by the prediction target information from the plurality of second conditions. (2) Extract _k.

[0074] (Step S123) Subsequently, in step S123, the prediction unit 12 determines whether the second condition c extracted in step S122 is satisfied. (2) The second weight vector w associated with k (2) Select _k.

[0075] (Step S124) Subsequently, in step S124, the prediction unit 12 calculates the prediction results f i Calculate (x).

[0076] (Step S125) Subsequently, in step S125, the prediction unit 12 calculates the second weight vector w selected in step S123. (2) Using _k, the integrated prediction result y^ is

number

[0077] For classification tasks, for example, i (x) represents a vector of the number of classes, and f i The predicted probability for each class label i is expressed by (x). The prediction unit 12 can then calculate the integrated predicted probability using an equation similar to the above equation (15). When finally determining a label as a predicted value, the prediction unit 12 determines the class label with the highest probability as the label as the predicted value.

[0078] (Step S126) In step S126, the prediction unit 12 determines whether the set X includes other prediction target information for which an integrated prediction result has not yet been calculated. If other prediction target information is included (YES in step S126), the process from step S122 is repeated for the other prediction target information. If other prediction target information is not included, the integrated prediction result y^ calculated in step S125 is output, and the process ends. Note that the next weight update process S11 may be performed using evaluation information that includes model input information included in the prediction target information used in the prediction process S12 as model input information for evaluation. However, as in exemplary embodiment 1, the weight update process S11 and the prediction process S12 may be performed independently of each other, and the order and timing of execution of each process are not specified.

[0079] (Effect of prediction device 1A) According to the prediction device 1A configured as described above, as with the prediction device 1 according to exemplary embodiment 1, some or all of the multiple first weight vectors and some or all of the multiple second weight vectors can be updated with high accuracy by not only based on the evaluation results of each model evaluated with reference to the evaluation information, but also based on the evaluation information referenced to obtain the evaluation results. Furthermore, when making predictions, weight vectors selected from the multiple first weight vectors and multiple second weight vectors updated in this manner based on the prediction target information are used, thereby achieving the effect of enabling highly accurate ensemble predictions even when the distribution of model input information included in the prediction target information changes locally.

[0080] Furthermore, the second weight vector may be a vector whose components are weights assigned to each of the plurality of first weight vectors, and the plurality of first weight vectors may include a weight vector associated with a condition obtained by combining any two or more conditions included in the plurality of first conditions.

[0081] By adopting such a configuration, even if the first condition c (1) jEven if the settings of are not necessarily appropriate, it is possible to perform suitable ensemble prediction. This effect will be explained below using a more specific example.

[0082] For example, in the example shown in FIG. 4, the first condition c (1) 2. Suppose that there is almost no evaluation information that satisfies the criteria "sunny holiday." In this case, the second weight vector w (2) k In a configuration that does not use the first condition c (1) The first weight vector w corresponding to 2 (1) 2 will be updated less frequently, which may result in a problem of reduced accuracy of ensemble prediction.

[0083] On the other hand, in reality, the first condition c (1) 1. Evaluation information that satisfies "sunny weekday" and the first condition c (1) 2. It is possible that the evaluation information that satisfies the condition "sunny weekday" was linked to the first condition c. (1) 3. Evaluation information that satisfies "rainy weekday" and the first condition c (1) It is possible that the first condition was linked to the evaluation information that satisfied the condition 4 "rainy weekday" (the distribution changed in conjunction with this). In such a case, if the first condition had been set to a rough condition such as "sunny" or "rainy," the update frequency of the first weight vector would have decreased, and the problem of the accuracy of the ensemble forecast would not have occurred.

[0084] In this way, in a configuration in which conditions and weights are associated with each other, how the conditions are set can have a significant effect on prediction accuracy. However, in this exemplary embodiment, as described above, the multiple first weight vectors w (1) j The second weight vector w is composed of weights assigned to each of (2) k and the plurality of first weight vectors w (1) jThe configuration is adopted in which the weight vector associated with a condition obtained by combining any two or more conditions included in the plurality of first conditions may be included in the weight vector. More specifically, in the example shown in FIG. 4, the plurality of first conditions c (1) j (j=1, ,7) First condition c (1) 1. "Sunny weekday" and the first condition c (1) 2. Condition c obtained by integrating with "sunny holiday" (1) 5. Sunny First condition c (1) 3. "Rainy weekday" and the first condition c (1) 4. Condition c obtained by integrating with "rainy holiday" (1) 6. Rain Therefore, the first condition c (1) 1~c (1) Even if the setting of 4 is not necessarily appropriate, it is possible to suppress a decrease in the update frequency of the first weight vector and to suppress a decrease in the accuracy of the ensemble prediction.

[0085] Also, the second weight vector w (2) k By adopting the second weight vector w (2) k This also has the secondary effect of being able to obtain information about the relationship between different conditions from the components of (2) The second vector associated with 2 w (2) 2= ​​(w (2) 2,1 ,···,w (2) 2,Nweight(1) ) In this case, component w (2) 2,1 If the value of is 1 and the other components are almost zero, the second condition c (2) 2. "Sunny Holiday" and the first condition c (2) 1 It can be inferred that a similar change in distribution occurs between "sunny weekdays" and "sunny weekdays."

[0086] Third Exemplary Embodiment A third exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0087] (Prediction Device 1B) 7 is a block diagram showing the configuration of a prediction device 1B according to this exemplary embodiment. As shown in FIG. 7, the prediction device 1B according to this exemplary embodiment is the same as the prediction device 1 according to the exemplary embodiment 2. In addition to the components of A, it includes a display information generating unit 15 and an input / output unit 16.

[0088] The display information generating unit 15 generates display information by referring to the integrated prediction result derived by the predicting unit 12, the model pool MP, and the first and second weight vectors.

[0089] The input / output unit 16 is configured to include at least one of an input / output device such as a keyboard, a mouse, a display, a printer, a touch panel, etc. Alternatively, the input / output unit 16 may be configured to have an input / output device such as a keyboard, a mouse, a display, a printer, a touch panel, etc. connected to it. In this configuration, the input / output unit 16 accepts various types of information input to the prediction device 1B from the connected input device. Furthermore, the input / output unit 16 outputs various types of information to the connected output device. An interface such as a USB (Universal Serial Bus) may be used as the input / output unit 16.

[0090] Fig. 8 shows an example of display information generated by the display information generating unit 15 and displayed via the input / output unit 16. As shown in Fig. 8, the display information includes, as an example, a plurality of data points obtained by embedding model input information (explanatory variables) in a low-dimensional space (a two-dimensional space in the case of Fig. 8).

[0091] In the example shown in FIG. 8, data points indicated by the model input information are indicated using markers of different shapes for each condition that the model input information satisfies. For example, data points indicated using circular markers are indicated by the second condition c (2) The data points indicated by the model input information (explanatory variables) satisfying the second condition c are shown. (2) The data points indicated by the model input information (explanatory variables) satisfying 2 are shown.

[0092] The prediction unit 12 may refer to the second weight vector to identify a set of conditions among the multiple conditions that are causing changes in conjunction with each other, and reflect the identified sets in the display information. The example of Fig. 8 shows a case where the prediction unit 12 finds that the conditions satisfied by the model input information indicated by the circular data points and the conditions satisfied by the model input information indicated by the diamond data points are linked, and the display information generation unit 15 includes a boundary line CONT surrounding these data points in the display information.

[0093] 8, the input / output unit 16 may be configured to display a cursor CSR that can be operated by the user, allowing each data point to be selected. The display information generation unit 15 may be configured to generate additional information to be presented to the user based on an input from the user.

[0094] (Variation 1) In the above-described exemplary embodiment, the first weight vector and the first condition are associated one-to-one, and the second weight vector and the second condition are associated one-to-one. However, this is not limiting, and at least one of the multiple first weight vectors may be associated with two or more of the multiple first conditions. Similarly, at least one of the multiple second weight vectors may be associated with two or more of the multiple second conditions.

[0095] For example, assume that the first condition A and the first condition B are associated with the same first weight vector. In this case, the weight update unit 11 may update the first weight vector based on an evaluation result EA obtained by evaluating the performance of each model f_i using evaluation information that satisfies the first condition A, and an evaluation result EB obtained by evaluating the performance of each model f_i using evaluation information that satisfies the first condition B. The weight update unit 11 may update the first weight vector using, for example, a statistical value (such as an average value) calculated from the evaluation results EA and the evaluation results EB, but is not limited to this.

[0096] Furthermore, at least two or more of the plurality of first weight vectors may be associated with any one of the plurality of first conditions, and similarly, at least two or more of the plurality of second weight vectors may be associated with any one of the plurality of second conditions.

[0097] For example, suppose that the first weight vector A and the first weight vector B are associated with the same condition. In this case, the weight update unit 11 may update the first weight vector A and the first weight vector B based on an evaluation result obtained by evaluating the performance of each model f_i using evaluation information that satisfies the condition. Furthermore, when the prediction target information satisfies the condition, the prediction unit 12 may obtain an integrated prediction result using a weight vector (for example, a vector whose elements are the average values ​​of the elements) calculated from the first weight vector A and the first weight vector B associated with the condition.

[0098] For example, if the first condition is set to condition c_1 "weekday" and condition c_2 "holiday," and the first weight vector is set to weight vectors w_1, w_2, and w_3, the weight vectors w_1 and w_2 may be associated with condition c_1 "weekday," and the weight vectors w_2 and w_3 may be associated with condition c_2 "holiday." In other words, there may be two weight vectors corresponding to weekdays and two weight vectors corresponding to holidays. Alternatively, one weight vector may correspond to both weekdays and holidays.

[0099] (Variation 2) In the above-described exemplary embodiment, the first conditions c (1) _j, and multiple second conditions c (2) _k may be determined based on a rule given by a user, for example. (1) _j, and multiple second conditions c (2) _k may be determined by clustering a plurality of pieces of prediction target information given in advance. For example, a plurality of pieces of prediction target information given in advance may be clustered using a hard clustering method such as the K-means method or hierarchical clustering so that each piece of prediction target information belongs to one cluster. In this case, belonging to each of the plurality of clusters indicated by the clustering result satisfies each condition c. (1) _j or c (2) _k. Also, multiple first conditions c (1) _j, and multiple second conditions c (2) Some or all of k may change during operation.

[0100] (Variation 3) In addition, in the second modification, for example, a soft clustering method such as the Fuzzy C-means method or the Gaussian mixture model may be used for the preliminary clustering. In this case, for example, a membership value indicating the degree to which the prediction target information belongs to each of a plurality of clusters is calculated. In this case, too, belonging to each of a plurality of clusters indicated by the clustering result is determined by each condition c_j or c(2) It may be defined as _k.

[0101] (Variation 4) As in the third modification, multiple first conditions c are obtained by soft clustering. (1) _j or the second condition c (2) When k is determined, the weight update unit 11 may be modified as follows: The weight update unit 11 may calculate the degree to which the evaluation information satisfies each of a plurality of conditions, and update some or all of the plurality of weight vectors according to the degree to which each condition, either the first condition or the second condition, is satisfied.

[0102] For example, as a method for calculating the degree to which the evaluation information satisfies each of a plurality of conditions, a soft clustering method can be applied, as in Modification 3. For example, suppose that the model input information includes store surroundings images without including weekday / holiday labels. In this case, the weight update unit 11 may calculate, from the store surroundings images, the degree to which the image was taken on a weekday (the degree to which the condition c_1 is satisfied) and the degree to which the image is a holiday (the degree to which the condition c_2 is satisfied). For example, as a specific example, suppose that for certain evaluation information, the degree to which the condition c_1 "weekday" is satisfied is "0.3" and the degree to which the condition c_2 "holiday" is satisfied is "0.7". In this case, the weight update unit 11 calculates the evaluation results of the performance of each model f_i that referenced the evaluation information as the first weight vector w (1) Each element may be updated so that the difference between the pre-update and post-update differences of each element when the model f_i is applied to the model f_i as is is multiplied by 0.3. The weight update unit 11 may also update the first weight vector w (1) Each element may be updated so that the difference between the elements before and after the update when applied to _2 as is is multiplied by 0.7.

[0103] (Variation 5) As in Modification 3, the first condition c is satisfied by soft clustering. (1) _j or the second condition c (2)When _k is determined, the prediction unit 12 may be modified as follows: The prediction unit 12 may calculate the degree to which the prediction target information satisfies each of a plurality of conditions, and may integrate the prediction results predicted by each model f_i using an integrated weight vector obtained by integrating a plurality of weight vectors according to the degree to which each condition, either the first condition or the second condition, is satisfied.

[0104] For example, as a method for calculating the degree to which the prediction target information satisfies each of the multiple conditions, a soft clustering method can be applied as in the third modification. For example, (1) The degree to which j is satisfied (for example, the membership value of each cluster) is expressed by the following equation (16).

number

number

number

number

[0105] For example, it is assumed that the model input information includes images of the surrounding area of ​​a store without including weekday / holiday labels, and the prediction target information satisfies the first condition c (1) _1 "Weekday" degree of satisfaction "0.3" and the first condition c (1) In this case, the prediction unit 12 calculates the degree of satisfaction of the condition c_2 "holiday" as "0.7". (1) The weight vector w associated with _1 (1) _1 is assigned a weight of 0.3, and condition c (1) The weight vector w associated with _2 (1) By assigning a weight of 0.7 to _2, the weight vector w (1) _1 and w (1) The weighted average of _2 is the first integrated weight vector w (1) The same applies to the second weight vector. Furthermore, the prediction unit 12 calculates the prediction result y_i output from each model f_i by referring to the model input information included in the prediction target information, and calculates the first integrated weight vector w (1) and the second integrated weight vector w (2) The integrated prediction result is calculated by integrating using

[0106] (Variation 6) As partially described above, the present exemplary embodiment is not limited to an example in which the prediction task is a regression task, such as when the prediction target is, for example, the sales amount on a target date, and the prediction task may be a classification task. In this case, the prediction unit 12 may determine the weighted average of the prediction results (prediction probability of classification) of each model f_i as the integrated prediction result (prediction probability of classification). Alternatively, the prediction unit 12 may determine the integrated prediction result (classification) by a weighted majority vote of the prediction results (classification) of each model f_i.

[0107] (Application example 1) The following describes application examples of the above-described prediction devices 1, 1A, and 1B. In the following description, the application example of the prediction device 1A will be described, but application examples of the prediction devices 1 and 1B can also be realized in the same way.

[0108] For example, the prediction device 1A can be applied in the medical field. A specific example of the prediction device 1A applied in the medical field will be described with reference to FIG. 9. FIG. 9 is a schematic diagram showing a specific example of the prediction device 1A applied in the medical field. In this example, the prediction target is "bed occupancy rate one week from now," and the model input information is "weather, temperature, day of the week, disease name, most recent bed occupancy rate, and holiday / weekday label." Here, information about the weather is obtained from a weather providing server via an API (Application Programming Interface) through communication via a communication unit (not shown) provided in the prediction device 1A. Furthermore, information about temperature is obtained from a temperature sensor connected to the prediction device 1A via the input / output unit 16.

[0109] The prediction device 1A also stores models f_1, f_2, and f_3. The models f_1, f_2, and f_3 are generated in advance using training data by different learning algorithms. The model f_1 is generated by a deep neural network (DNN), the model f_2 is generated by a gradient boosted tree (GBDT), and the model f_3 is generated by linear regression.

[0110] The prediction device 1A then repeats the weight update process S11 and the prediction process S12 using the condition settings and the first and second weight vectors shown in Fig. 4. The "bed occupancy rate one week later," which is the integrated prediction result derived by the prediction device 1A, is input to a reservation management system connected to the prediction device 1A and is referenced for optimizing the number of hospital beds to be reserved. With this configuration, ensemble prediction can be performed with high accuracy even when the distribution of information related to the prediction target changes locally.

[0111] Also, as described above, a plurality of first conditions c (1) j(j=1, ,7), First condition c (1) 1. "Sunny weekday" and the first condition c (1) 2. Condition c obtained by integrating with "sunny holiday" (1) 5. Sunny First condition c (1) 3. "Rainy weekday" and the first condition c (1) 4. Condition c obtained by integrating "rainy holiday" and (1) 6. Rain By including the above, even if the condition settings are not necessarily appropriate, it is possible to suppress a decrease in the frequency of updating the weight vectors and a decrease in the accuracy of the ensemble prediction. Therefore, since it is possible to output highly accurate integrated prediction results, the prediction device 1A can preferably support the decision-making of hospital staff (healthcare professionals) such as doctors involved in bed management.

[0112] (Application example 2) As another example, the prediction device 1A can be applied to demand forecasting. A specific example of the prediction device 1A applied to demand forecasting will be described with reference to FIG. 10. FIG. 10 is a schematic diagram showing a specific example of the prediction device 1A applied to demand forecasting. In this example, the prediction target is "next day's sales," and the model input information is "weather, temperature, day of the week, product classification, moving average, and holiday / weekday label." Here, information about the weather is obtained from a weather providing server via an API (Application Programming Interface) through communication via a communication unit (not shown) provided in the prediction device 1A. Furthermore, information about temperature is obtained from a temperature sensor connected to the prediction device 1A via the input / output unit 16.

[0113] The prediction device 1A also stores models f_1, f_2, and f_3. The models f_1, f_2, and f_3 are generated in advance using training data by different learning algorithms. The model f_1 is generated by a deep neural network (DNN), the model f_2 is generated by a gradient boosted tree (GBDT), and the model f_3 is generated by linear regression.

[0114] The prediction device 1A then repeats the weight update process S11 and the prediction process S12 using the condition settings and the first and second weight vectors shown in Fig. 4. The "next day's sales," which is the integrated prediction result derived by the prediction device 1A, is input to a reservation management system connected to the prediction device 1A and is referenced for optimizing the number of orders. This configuration enables ensemble prediction with high accuracy even when the distribution of information related to the prediction target changes locally.

[0115] Here, as described above, a plurality of first conditions c (1) j (j=1, ,7), First condition c (1) 1. "Sunny weekday" and the first condition c (1) 2. Condition c obtained by integrating with "sunny holiday" (1) 5. Sunny First condition c (1) 3. "Rainy weekday" and the first condition c (1) 4. Condition c obtained by integrating with "rainy holiday" (1) 6. Rain By including these parameters, as described above, even if the conditions are not necessarily set appropriately, it is possible to prevent a decrease in the frequency of updating the weight vectors and a decrease in the accuracy of the ensemble prediction. As a result, it is possible to output highly accurate integrated prediction results.

[0116] [Software implementation example] Some or all of the functions of the prediction devices 1, 1A, and 1B (hereinafter also referred to as "each of the above devices") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0117] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 11. Figure 11 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0118] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0119] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0120] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0121] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0122] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0123] (Appendix A1) a weight update means for updating some or all of the plurality of first weight vectors and some or all of the plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of the plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; a prediction means for integrating prediction results predicted by each model by referring to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputting an integrated prediction result; A prediction device comprising:

[0124] (Appendix A2) each of the plurality of first weight vectors is associated with at least one of a plurality of first conditions that the prediction target information may satisfy and that the evaluation information may satisfy; The weight updating means updates a first weight vector associated with a first condition satisfied by the evaluation information, among the plurality of first conditions, based on the evaluation result. 10. The prediction device according to claim A1.

[0125] (Appendix A3) each of the plurality of second weight vectors is associated with at least one of a plurality of second conditions that the prediction target information may satisfy and that the evaluation information may satisfy; the weight update means updates a second weight vector associated with a second condition satisfied by the evaluation information among the plurality of second conditions based on the evaluation result; the prediction means selects a second weight vector associated with a second condition satisfied by the prediction target information from among the plurality of second conditions; 10. The prediction device according to claim A2.

[0126] (Appendix A4) The second weight vector is a vector whose components are weights assigned to each of the plurality of first weight vectors. 10. The prediction device according to claim A3.

[0127] (Appendix A5) For a certain piece of prediction target information, there are a plurality of first conditions that are satisfied by the prediction target information, For a certain piece of prediction target information, the second condition satisfied by the prediction target information is determined to be one. A prediction device according to appendix A3 or A4 (Appendix A6) The plurality of first weight vectors include a weight vector associated with a condition obtained by combining any two or more conditions included in the plurality of first conditions. 10. The prediction device according to claim 9, wherein the first and second inputs are input to the prediction device.

[0128] (Appendix A7) The prediction target information further includes, in addition to the model input information, additional information that is not input to each model; the evaluation information further includes the additional information for evaluation in addition to the model input information for evaluation; 10. The prediction device according to any one of appendices A1 to A6.

[0129] (Appendix A8) At least one of the plurality of models is a machine learning model. 10. The prediction device according to any one of appendices A1 to A7.

[0130] [Appendix B] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0131] (Appendix B1) a weight update process in which at least one processor updates some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; a prediction process in which the at least one processor integrates prediction results predicted by each model by referring to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputs an integrated prediction result; A prediction method comprising:

[0132] (Appendix B2) each of the plurality of first weight vectors is associated with at least one of a plurality of first conditions that the prediction target information may satisfy and that the evaluation information may satisfy; In the weight update process, the at least one processor updates a first weight vector associated with a first condition satisfied by the evaluation information among the plurality of first conditions based on the evaluation result. Forecasting method described in Appendix B1.

[0133] (Appendix B3) each of the plurality of second weight vectors is associated with at least one of a plurality of second conditions that the prediction target information may satisfy and that the evaluation information may satisfy; In the weight update process, the at least one processor updates a second weight vector associated with a second condition satisfied by the evaluation information among the plurality of second conditions based on the evaluation result; In the prediction process, the at least one processor selects a second weight vector associated with a second condition satisfied by the prediction target information from among the plurality of second conditions. Forecasting method described in Appendix B2.

[0134] (Appendix B4) The second weight vector is a vector whose components are weights assigned to each of the plurality of first weight vectors. Forecasting methodology described in Appendix B3.

[0135] (Appendix B5) For a certain piece of prediction target information, there are a plurality of first conditions that are satisfied by the prediction target information, For a certain piece of prediction target information, the second condition satisfied by the prediction target information is determined to be one. A prediction device according to appendix B3 or B4 (Appendix B6) The plurality of first weight vectors include a weight vector associated with a condition obtained by combining any two or more conditions included in the plurality of first conditions. 10. The prediction method of any one of Appendices B2 to B5.

[0136] (Appendix B7) The prediction target information further includes, in addition to the model input information, additional information that is not input to each model; the evaluation information further includes the additional information for evaluation in addition to the model input information for evaluation; 10. The prediction method of any one of Appendices B1 to B6.

[0137] (Appendix B8) At least one of the plurality of models is a machine learning model. 10. The prediction method of any one of Appendices B1 to B7.

[0138] [Appendix C] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0139] (Appendix C1) A program that causes a computer to function as a prediction device, The computer a weight update means for updating some or all of the plurality of first weight vectors and some or all of the plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of the plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; a prediction means for integrating prediction results predicted by each model by referring to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputting an integrated prediction result; A prediction program that functions as a

[0140] (Appendix C2) each of the plurality of first weight vectors is associated with at least one of a plurality of first conditions that the prediction target information may satisfy and that the evaluation information may satisfy; The weight updating means updates a first weight vector associated with a first condition satisfied by the evaluation information, among the plurality of first conditions, based on the evaluation result. The prediction program described in Appendix C1.

[0141] (Appendix C3) each of the plurality of second weight vectors is associated with at least one of a plurality of second conditions that the prediction target information may satisfy and that the evaluation information may satisfy; the weight update means updates a second weight vector associated with a second condition satisfied by the evaluation information among the plurality of second conditions based on the evaluation result; the prediction means selects a second weight vector associated with a second condition satisfied by the prediction target information from among the plurality of second conditions; 1. The prediction program described in Appendix C2.

[0142] (Appendix C4) The second weight vector is a vector whose components are weights assigned to each of the plurality of first weight vectors. The prediction program described in Appendix C3.

[0143] (Appendix C5) For a certain piece of prediction target information, there are a plurality of first conditions that are satisfied by the prediction target information, For a certain piece of prediction target information, the second condition satisfied by the prediction target information is determined to be one. A prediction device according to appendix C3 or C4 (Appendix C6) The plurality of first weight vectors include a weight vector associated with a condition obtained by combining any two or more conditions included in the plurality of first conditions. 10. The prediction program of claim 9, wherein the prediction program is a program for predicting a prediction state of a target object.

[0144] (Appendix C7) The prediction target information further includes, in addition to the model input information, additional information that is not input to each model; the evaluation information further includes the additional information for evaluation in addition to the model input information for evaluation; 10. The prediction program of any one of appendices C1 to C6.

[0145] (Appendix C8) At least one of the plurality of models is a machine learning model. 10. The prediction program of any one of appendices C1 to C7.

[0146] [Appendix D] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0147] (Appendix D1) at least one processor, a weight update process for updating some or all of the plurality of first weight vectors and some or all of the plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of the plurality of models with reference to evaluation information including model input information for evaluation, and the evaluation information; a prediction process for outputting an integrated prediction result obtained by integrating prediction results predicted by each model with reference to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information; A prediction device that performs the following.

[0148] The prediction device may further include a memory, and the memory may store a program for causing the at least one processor to execute each of the processes.

[0149] (Appendix D2) each of the plurality of first weight vectors is associated with at least one of a plurality of first conditions that the prediction target information may satisfy and that the evaluation information may satisfy; In the weight update process, the at least one processor updates a first weight vector associated with a first condition satisfied by the evaluation information among the plurality of first conditions based on the evaluation result. 10. The prediction device according to claim D1.

[0150] (Appendix D3) each of the plurality of second weight vectors is associated with at least one of a plurality of second conditions that the prediction target information may satisfy and that the evaluation information may satisfy; In the weight update process, the at least one processor updates a second weight vector associated with a second condition satisfied by the evaluation information among the plurality of second conditions based on the evaluation result; In the prediction process, the at least one processor selects a second weight vector associated with a second condition satisfied by the prediction target information from among the plurality of second conditions. 10. The prediction device of claim D2.

[0151] (Appendix D4) The second weight vector is a vector whose components are weights assigned to each of the plurality of first weight vectors. 10. The prediction device of claim 9, wherein the prediction device comprises:

[0152] (Appendix D5) For a certain piece of prediction target information, there are a plurality of first conditions that are satisfied by the prediction target information, For a certain piece of prediction target information, the second condition satisfied by the prediction target information is determined to be one. The prediction device according to appendix D3 or D4 (Appendix D6) The plurality of first weight vectors include a weight vector associated with a condition obtained by combining any two or more conditions included in the plurality of first conditions. 10. The prediction device of claim 9, wherein the first and second inputs are input to the prediction device.

[0153] (Appendix D7) The prediction target information further includes, in addition to the model input information, additional information that is not input to each model; the evaluation information further includes the additional information for evaluation in addition to the model input information for evaluation; 10. The prediction device according to any one of appendices D1 to D6.

[0154] (Appendix D8) At least one of the plurality of models is a machine learning model. 10. The prediction device according to any one of appendices D1 to D7.

[0155] [Appendix E] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0156] (Appendix E1) A program that causes a computer to function as a prediction device, The computer, a weight update process for updating some or all of the plurality of first weight vectors and some or all of the plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of the plurality of models with reference to evaluation information including model input information for evaluation, and the evaluation information; a prediction process for outputting an integrated prediction result obtained by integrating prediction results predicted by each model with reference to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information; A non-transitory recording medium on which a prediction program is recorded, which causes the program to execute the prediction program. [Explanation of symbols]

[0157] 1, 1A, 1B Prediction Device 11 Prediction Department 12 Update section 13 Model memory section 14 Weight storage unit C1 processor C2 Memory

Claims

1. a weight update means for updating some or all of the plurality of first weight vectors and some or all of the plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of the plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; a prediction means for integrating prediction results predicted by each model with reference to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputting an integrated prediction result; A prediction device comprising:

2. each of the plurality of first weight vectors is associated with at least one of a plurality of first conditions that the prediction target information may satisfy and that the evaluation information may satisfy; The weight updating means updates a first weight vector associated with a first condition satisfied by the evaluation information, among the plurality of first conditions, based on the evaluation result. The prediction device according to claim 1 .

3. each of the plurality of second weight vectors is associated with at least one of a plurality of second conditions that the prediction target information may satisfy and that the evaluation information may satisfy; the weight update means updates a second weight vector associated with a second condition satisfied by the evaluation information among the plurality of second conditions based on the evaluation result; the prediction means selects a second weight vector associated with a second condition satisfied by the prediction target information from among the plurality of second conditions; The prediction device according to claim 2 .

4. The second weight vector is a vector whose components are weights assigned to each of the plurality of first weight vectors. The prediction device according to claim 3 .

5. For a certain piece of prediction target information, there are a plurality of first conditions that are satisfied by the prediction target information, For a certain piece of prediction target information, the second condition satisfied by the prediction target information is determined to be one. The prediction device according to claim 3 or 4.

6. The plurality of first weight vectors include a weight vector associated with a condition obtained by combining any two or more conditions included in the plurality of first conditions. The prediction device according to claim 2 .

7. The prediction target information further includes, in addition to the model input information, additional information that is not input to each model; the evaluation information further includes the additional information for evaluation in addition to the model input information for evaluation; The prediction device according to claim 1 .

8. At least one of the plurality of models is a machine learning model. The prediction device according to claim 1 .

9. a weight update process in which at least one processor updates some or all of a plurality of first weight vectors and some or all of a plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of a plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; a prediction process in which the at least one processor integrates prediction results predicted by each model with reference to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputs an integrated prediction result; A prediction method comprising:

10. A program that causes a computer to function as a prediction device, the program comprising: a weight update means for updating some or all of the plurality of first weight vectors and some or all of the plurality of second weight vectors based on an evaluation result obtained by evaluating the performance of each of the plurality of models with reference to evaluation information including model input information for evaluation and true values ​​corresponding to the model input information, and the evaluation information; a prediction means for integrating prediction results predicted by each model with reference to model input information included in prediction target information related to a prediction target, using a weight vector selected from the plurality of first weight vectors and the plurality of second weight vectors based on the prediction target information, and outputting an integrated prediction result; A prediction program that functions as a

Images