Information processing device, learning method, and learning program
By using a decision list with top K decision rules, the prediction performance of decision lists is enhanced, making them competitive with deep neural networks and random forests.
Patent Information
- Application Number
- JP2023527153
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-06-07
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-06-07
AI Technical Summary
The prediction performance of decision lists is inferior to black box models like deep neural networks and random forests, limiting their practical application.
An information processing device and method that utilizes a decision list comprising multiple decision rules, calculating prediction results using the top K decision rules whose conditions are satisfied, and determining an optimal decision list based on these results to enhance prediction performance.
Improves the prediction performance of decision lists compared to conventional methods, enabling them to compete with black box models.
Smart Images

Figure 0007586319000022 
Figure 0007586319000023 
Figure 0007586319000024
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device or the like that outputs a decision list through machine learning. [Background technology]
[0002] Predictions made by artificial intelligence (AI) using black-box models such as deep neural networks and random forests have the drawback of being unable to explain the basis for the predictions.
[0003] For this reason, a prediction model called a decision list has been drawing renewed attention as one type of AI that can explain the basis for prediction. As described in Non-Patent Document 1 below, a decision list is a list consisting of multiple If-Then rules. In predictions using a decision list, among the rules for which an observation satisfies a condition (the "If" in an If-Then rule), a prediction is made by applying the rule that is at the top of the decision list. For this reason, the prediction result can be explained by one rule, and it is also easy for humans to understand how the rule was selected. In this way, a decision list has the advantage of being able to explain the basis for a prediction. [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Cynthia Rudin, Seyda Ertekin, “Learning customized and optimized lists of rules with mathematical programming”, Math. Program. Comput., 2018. Summary of the Invention [Problem to be solved by the invention]
[0005] However, the technique of Non-Patent Document 1 has a problem that the prediction performance is inferior to black box models such as deep neural networks and random forests. The present invention aims to provide an information processing device or the like that can improve the prediction performance of predictions using decision lists. [Means for solving the problem]
[0006] An information processing device according to one aspect of the present invention comprises a prediction means for calculating a prediction result using the predicted values of the top K (K is a natural number greater than or equal to 2) decision rules whose training examples included in a training example set satisfy the conditions, out of the decision rules included in a decision list consisting of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules that combine a condition and a predicted value when the condition is satisfied, and a list determination means for determining a decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of a plurality of decision lists generated from the decision rule set.
[0007] An information processing device according to one aspect of the present invention comprises an input data acquisition means for acquiring input data to be predicted, and a prediction means for calculating a prediction result using the predicted values of the top K (K is a natural number equal to or greater than 2) decision rules among decision rules included in a decision list consisting of a plurality of decision rules each combining a condition and a predicted value when the condition is satisfied, for which the input data satisfies the condition.
[0008] A learning method according to one aspect of the present invention includes at least one processor calculating a prediction result using the predicted values of the top K (K is a natural number greater than or equal to 2) decision rules whose training examples included in a training example set satisfy the conditions, out of the decision rules included in a decision list consisting of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules that combine a condition and a predicted value when the condition is satisfied, and determining a decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set.
[0009] A learning program according to one aspect of the present invention causes a computer to function as: a prediction means for calculating a prediction result using the predicted values of the top K (K is a natural number greater than or equal to 2) decision rules whose training examples included in a training example set satisfy the conditions, out of the decision rules included in a decision list consisting of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules that combine a condition and a predicted value when the condition is satisfied; and a list determination means for determining a decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set. Effect of the Invention
[0010] According to one aspect of the present invention, it is possible to improve the prediction performance of predictions using decision lists. [Brief description of the drawings]
[0011] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment of the present invention. [Diagram 2] FIG. 2 is a flow chart showing the flow of a learning method and a prediction method according to the first exemplary embodiment of the present invention. [Diagram 3] FIG. 11 is a diagram showing an overview of a learning method according to an exemplary embodiment 2 of the present invention. [Figure 4]FIG. 11 is a block diagram showing an example of the configuration of an information processing device according to an exemplary embodiment 2 of the present invention. [Diagram 5] 4 is a flowchart showing a flow of a learning method executed by the information processing device. [Figure 6] FIG. 11 is a diagram showing an overview of a learning method according to an exemplary embodiment 3 of the present invention. [Figure 7] FIG. 11 is a block diagram showing an example of the configuration of an information processing device according to an exemplary embodiment 3 of the present invention. [Figure 8] 4 is a flowchart showing a flow of a learning method executed by the information processing device. [Figure 9] FIG. 11 is a block diagram showing an example of the configuration of an information processing device according to an exemplary embodiment 4 of the present invention. [Figure 10] FIG. 1 is a block diagram showing a configuration of an information processing device according to a reference example. [Figure 11] FIG. 1 is a diagram illustrating an example of a computer that executes instructions of a program, which is software that realizes each function of an information processing device according to each exemplary embodiment and reference example of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Example 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later.
[0013] (Configuration of information processing device 1) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configurations of the information processing device 1 and the information processing device 2. The information processing device 2 will be described later. As shown in the figure, the information processing device 1 includes a prediction unit (prediction means) 11 and a list determination unit (list determination means) 12.
[0014] The prediction unit 11 calculates a prediction result using the predicted values of the top K (K is a natural number greater than or equal to 2) decision rules whose training examples included in the training example set satisfy the conditions, out of the decision rules included in a decision list consisting of multiple decision rules extracted from a decision rule set, which is a set of decision rules that combine conditions and predicted values when the conditions are satisfied.
[0015] The list determination unit 12 determines a decision list to be output based on the prediction results calculated for each training example included in the training example set, for each of the multiple decision lists generated from the decision rule set.
[0016] As described above, the information processing device 1 according to this exemplary embodiment is configured to include a prediction unit 11 that calculates a prediction result using the predicted values of the top K (K is a natural number equal to or greater than 2) decision rules among the decision rules included in a decision list consisting of a plurality of decision rules extracted from a decision rule set, for which the training examples included in the training example set satisfy a condition, and a list determination unit 12 that determines a decision list to be output based on the prediction results calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set.
[0017] According to the above configuration, a decision list to be output is determined based on a prediction result calculated using the top K predicted values (K is a natural number equal to or greater than 2) that satisfy a condition. This makes it possible to determine a decision list to be output in order to perform prediction using the top K predicted values. According to such a decision list, prediction performance can be expected to be improved compared to conventional methods that use only the top predicted value in a prediction list. In other words, according to the above configuration, it is possible to achieve an effect of improving the prediction performance of predictions using a decision list.
[0018] (Configuration of information processing device 2) Next, a description will be given of the information processing device 2. As shown in Fig. 1, the information processing device 2 includes an input data acquisition unit (input data acquisition means) 21 and a prediction unit (prediction means) 22.
[0019] The input data acquisition unit 21 acquires input data to be predicted.
[0020] The prediction unit 22 calculates a prediction result using the predicted values of the top K (K is a natural number greater than or equal to 2) decision rules whose conditions are satisfied by the input data, from among the decision rules included in a decision list consisting of multiple decision rules that combine conditions and predicted values when the conditions are satisfied.
[0021] As described above, the information processing device 2 according to this exemplary embodiment is configured to include an input data acquisition unit 21 that acquires input data to be predicted, and a prediction unit 22 that calculates a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules for which the input data satisfies the conditions, among the decision rules included in a decision list consisting of multiple decision rules combining conditions and predicted values when the conditions are satisfied. This provides an effect of improving prediction performance compared to a conventional method that uses only the top predicted value in the prediction list.
[0022] (program) The above-mentioned functions of the information processing device 1 can also be realized by a learning program. The learning program according to this exemplary embodiment causes a computer to function as a prediction means for calculating a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules whose training examples included in a training example set satisfy the conditions, among the decision rules included in a decision list composed of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied, and as a list determination means for determining a decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set. Therefore, according to the learning program according to this exemplary embodiment, it is possible to obtain an effect that the prediction performance of prediction using a decision list can be improved.
[0023] The above-mentioned functions of the information processing device 2 can also be realized by a prediction program. The prediction program according to this exemplary embodiment causes a computer to function as an input data acquisition means for acquiring input data to be predicted, and a prediction means for calculating a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules for which the input data satisfies the conditions, among the decision rules included in a decision list consisting of a plurality of decision rules each of which combines a condition with a predicted value when the condition is satisfied. Therefore, according to the prediction program according to this exemplary embodiment, it is possible to obtain an effect that prediction performance can be improved compared to the conventional method of using only the top predicted value in the prediction list.
[0024] (Learning method flow) The flow of the learning method according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the learning method and the prediction method. The prediction method will be described later.
[0025] The execution entity of each step in the learning method of FIG. 2 may be a processor provided in the information processing device 1, or may be a processor provided in another device, or each step may be executed by a processor provided in a different device.
[0026] In S11, at least one processor calculates a prediction result using the predicted values of the top K (K is a natural number greater than or equal to 2) decision rules whose training examples in the training example set satisfy the conditions, out of the decision rules included in a decision list consisting of multiple decision rules extracted from a decision rule set, which is a set of decision rules that combine conditions and predicted values when the conditions are satisfied.
[0027] In S12, at least one processor determines a decision list to be output based on the prediction results calculated for each training example included in the training example set, for each of a plurality of decision lists generated from the decision rule set.
[0028] As described above, the learning method according to the present exemplary embodiment includes a configuration in which at least one processor calculates a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules whose training examples included in a training example set satisfy the conditions, among the decision rules included in a decision list composed of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied, and determines a decision list to be output based on the prediction results calculated for each training example included in the training example set for each of the plurality of decision lists generated from the decision rule set. Therefore, the learning method according to the present exemplary embodiment has an effect of improving the prediction performance of prediction using a decision list.
[0029] (Prediction method flow) Next, a flow of the prediction method according to this exemplary embodiment will be described with reference to Fig. 2. Note that the execution subject of each step in the prediction method in Fig. 2 may be a processor included in the information processing device 2, or may be a processor included in another device, or each step may be a processor provided in a different device.
[0030] In S21, at least one processor obtains input data to be predicted.
[0031] In S22, at least one processor calculates a prediction result using the prediction values of the top K (K is a natural number equal to or greater than 2) decision rules whose conditions are satisfied by the input data, from among the decision rules included in a decision list consisting of a plurality of decision rules each of which combines a condition with a predicted value when the condition is satisfied.
[0032] As described above, the prediction method according to the present exemplary embodiment includes a configuration in which at least one processor obtains input data to be predicted, and calculates a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules that satisfy the conditions of the decision rules included in a decision list consisting of a plurality of decision rules that combine conditions and prediction values when the conditions are satisfied. Therefore, according to the prediction method according to the present exemplary embodiment, it is possible to obtain an effect that prediction performance can be improved compared to the conventional method that uses only the top prediction value of the prediction list. Note that the prediction list used in the prediction method may be the prediction list determined in S12.
[0033] Exemplary embodiment 2 The second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate. This also applies to the third and subsequent exemplary embodiments.
[0034] (overview) 3 is a diagram showing an overview of the learning method according to the present exemplary embodiment. As in the first exemplary embodiment, the learning method according to the present exemplary embodiment also determines a decision list to be output, which is made up of a number of decision rules extracted from a decision rule set.
[0035] More specifically, in the learning method according to the present exemplary embodiment, multiple decision list candidates (hereinafter referred to as candidate lists) are generated from a decision rule set. Then, using each generated candidate list, prediction is performed for each training example included in a training example set. Then, based on the prediction result, a decision list to be output is determined from each candidate list.
[0036] For example, the decision rule set shown in FIG. R It contains R decision rules up to and including 1. Each decision rule corresponds to a condition (IF) and a predicted value (THEN) when the condition is met.
[0037] FIG. 3 shows the decision rules r1 to r R Among the candidate list generated using decision rules r4, r6, r2, ..., r R The condition of decision rule r4 is "x0>1.0 AND x2<2.0", and the predicted value is "80%". The condition of decision rule r6 is "x1>2.0", and the predicted value is "20%". The condition of decision rule r2 is "x2<3.0", and the predicted value is "70%". And the condition of decision rule r R The condition of is "TRUE" and the predicted value is "50%". R always outputs the same predicted value (50% in this example) for any input, and is called the default rule.
[0038] Using this candidate list, predictions are made for each training example included in the training example set. The training examples shown in FIG. 3 are associated with an observation ID, numerical values x0 to x2 indicating input, and a numerical value y indicating output. The input can also be said to be an observation value. The output y can also be said to be a label or correct answer data for the observation. Note that the observation value is not limited to a numerical value, and may be, for example, "TRUE" (meets a specified condition) and "FALSE" (does not meet a specified condition). Note that, although the unit of the output y is %, the unit of the output y is arbitrary as long as it is expressed as a real value.
[0039] Note that predictions using decision lists can be used to predict solutions to both regression problems and classification problems. In the case of a decision list that predicts solutions to regression problems, the output y will be a real value, as in the example in Figure 3. On the other hand, in the case of a decision list that predicts solutions to classification problems, the output y will be a probability vector that represents the probability of belonging to each class to which it is classified.
[0040] Now, suppose prediction is made for the training example with observation ID=0 in Figure 3. In this case, the decision rules are checked in order from the top to see whether the input values of the training example "x0=1.8, x1=1.5, x2=1.0" satisfy the conditions contained in the candidate list. This process is repeated until the number of decision rules that satisfy the conditions reaches K (K is a natural number equal to or greater than 2).
[0041] Here, it is assumed that K=2. In this case, as shown in Figure 3, the first decision rule r4 satisfies the condition, the next decision rule r6 does not, and the third decision rule r2 satisfies the condition, so the confirmation ends at this point. Then, the final prediction result is calculated using the prediction values of decision rules r4 and r6 that satisfy the condition.
[0042] For example, in the example in Figure 3, the final prediction result is the average (75%) of the prediction value "80%" of decision rule r4 and the prediction value "70%" of decision rule r6. The validity of this prediction result can be evaluated by comparing it with the value of the label y shown in the training example set. In addition, by performing the same process on each training example with observation ID "1" and onwards, the prediction accuracy of the candidate list for the entire training example set can be evaluated.
[0043] By performing the above-described process of evaluating the prediction accuracy of the candidate list for each of the multiple candidate lists, it is possible to identify the candidate list with the highest prediction accuracy and to determine that candidate list as the decision list to be output. This makes it possible to output a decision list that is composed of simple rules and has high prediction performance.
[0044] (Configuration of information processing device 3) 4 is a block diagram showing an example of the configuration of an information processing device 3 according to this exemplary embodiment. As shown in the figure, the information processing device 3 includes a control unit 30 that controls each unit of the information processing device 3, and a storage unit 31 that stores various data used by the information processing device 3. The information processing device 3 also includes an input unit 33 that accepts input to the information processing device 3, and an output unit 34 that allows the information processing device 3 to output data.
[0045] The control unit 30 includes a candidate generation unit 301, a prediction unit 302, a list determination unit 303, and an input data acquisition unit 304. In addition, the storage unit 31 stores a decision rule set 311, a training example set 312, and a decision list 313.
[0046] The decision rule set 311 is a set including a plurality of decision rules that can be used to generate a decision list, as described above. The training example set 312 is a set including a plurality of training examples that are used for learning, i.e., for determining an optimal decision list. Each training example is composed of a combination of an input x and an output y. The decision list 313 is a decision list that is determined by the list determination unit 303 to be output.
[0047] The candidate generation unit 301 generates a candidate list that is a candidate for a decision list by using the decision rules included in the decision rule set 311. More specifically, the candidate generation unit 301 generates a plurality of candidate lists that differ in at least one of the number of decision rules included and the arrangement order of the decision rules. For example, the candidate generation unit 301 may generate a candidate list of all patterns that can be generated by using the decision rules included in the decision rule set 311.
[0048] The prediction unit 302 calculates a prediction result using predicted values of the top K (K is a natural number of 2 or more) decision rules whose conditions are satisfied by the training examples included in the training example set 312, among the decision rules included in the candidate list generated by the candidate generation unit 301. After the list determination unit 303 determines the decision list to be output and stores it in the storage unit 31 as the decision list 313, the prediction unit 302 makes a prediction using the decision list 313.
[0049] The list determination unit 303 determines a decision list to be output for each training example included in the training example set 312, based on the prediction result calculated by the prediction unit 302 for each of the multiple candidate lists generated by the candidate generation unit 301. The decision list to be output is stored in the storage unit 31 as a decision list 313.
[0050] The input data acquisition unit 304 acquires input data to be predicted using the decision list 313. For this reason, the input data is data in the same format as the training example used for learning the decision list 313. For example, as in the example of Fig. 3, when using the decision list 313 output by learning using training examples whose inputs are x0, x1, and x2, the input data acquisition unit 304 acquires input data indicating at least any one of the values of x0, x1, and x2.
[0051] (Learning method flow) The flow of the learning method executed by the information processing device 3 will be described with reference to Fig. 5. Fig. 5 is a flow diagram showing the flow of the learning method executed by the information processing device 3.
[0052] In S31, the candidate generation unit 301 initializes the size L of the candidate list. Note that L indicates the number of decision rules included in the candidate list. The initial value of L may be the minimum value of L, for example, 1.
[0053] In S32, the candidate generator 301 generates a candidate list consisting of L decision rules. For example, the candidate generator 301 may generate the candidate list by arbitrarily extracting L decision rules from the decision rule set 311 and arbitrarily rearranging the L decision rules.
[0054] In S33, the prediction unit 302 uses the candidate list generated in S32 to calculate a prediction result for each training example included in the training example set 312. The prediction result is calculated using the top K prediction values that satisfy the conditions of the training example among the multiple decision rules included in the candidate list. For example, the prediction unit 302 may take the average of the top K prediction values as the prediction result.
[0055] In S34, the list determination unit 303 calculates the error between the output value y shown in the training example set 312 and the prediction result calculated in S33. Any method for calculating the error may be used, and the list determination unit 303 may calculate, for example, a squared error. In this case, the list determination unit 303 calculates the difference between the prediction result of the prediction unit 302 and the output value y, and squares it to obtain the error.
[0056] In S35, the list determination unit 303 judges whether the error for the candidate list of all patterns to be tried has been calculated. If the result of the judgment in S35 is NO, the process returns to S32, and a candidate list that has not been generated so far is generated. On the other hand, if the result of the judgment in S35 is YES, the process proceeds to S36.
[0057] All patterns to be tried may be determined in advance. For example, all patterns in a candidate list of size L that can be generated from the decision rules included in the decision rule set 311 may be set as trial targets.
[0058] In S36, the list determination unit 303 judges whether the current size L is smaller than the number |R| of decision rules included in the decision rule set 311. If the judgement in S36 is YES, the process proceeds to S37. In S37, the list determination unit 303 increments L by 1. Thereafter, the process returns to S32, and a candidate list is generated based on the incremented L. On the other hand, if the judgement in S36 is NO, the process proceeds to S38.
[0059] In S38, the list determination unit 303 determines the decision list to be output. Specifically, the list determination unit 303 determines the candidate list with the smallest error calculated in S34 as the decision list to be output. Then, the list determination unit 303 stores the determined decision list in the storage unit 31 as the decision list 313, and the process in FIG. 5 is completed.
[0060] Note that instead of generating candidate lists for all patterns for each value of size L, candidate lists for some patterns may be generated, and the candidate list with the smallest error among the candidate lists may be determined as the decision list to be output. In this case, the decision list determined to be output may not be the optimal decision list, but the time and amount of calculation required for learning can be reduced.
[0061] Also, the learning may be terminated when the error calculated in S34 becomes equal to or less than a predetermined threshold, and the candidate list with the error equal to or less than the threshold may be determined as the decision list to be output. In this case, the optimal decision list may not be selected as the decision list to be output, but the time and amount of calculation required for learning may be reduced.
[0062] (Prediction method flow) The prediction method executed by the information processing device 3 is the same as the prediction method shown in Fig. 2. Specifically, first, the input data acquisition unit 304 acquires input data to be predicted (S21). Next, the prediction unit 302 calculates prediction values of the top K decision rules whose conditions are satisfied by the input data acquired in S21 among the decision rules included in the decision list 313, and calculates a prediction result using these prediction values.
[0063] Exemplary embodiment 3 (overview) 6 is a diagram showing an overview of the learning method according to the present exemplary embodiment. As in the first and second exemplary embodiments, the learning method according to the present exemplary embodiment also determines a decision list to be output, which is made up of a number of decision rules extracted from a decision rule set.
[0064] More specifically, in the learning method according to the present exemplary embodiment, four variables, A j,u , D j,u,k , M j,u , and H i,k We also introduce a variable π u and δ u,j Introduce the following.
[0065] Although details will be described later, by introducing these variables, the decision list optimization problem can be made into an integer linear programming problem (hereinafter referred to as ILP: Integer Linear Programming). ILP can be solved efficiently and quickly using a known optimization solver, and the optimal decision list is determined by decoding the solution. For example, Gurobi or CPLEX can be applied as the optimization solver.
[0066] In addition, in this exemplary embodiment, a process of generating a training example set from a set of decision trees will also be described. Note that in the learning method according to this exemplary embodiment, it is not essential to generate a training example set from a set of decision trees, and the training example set used in the learning method is not limited to one generated from a set of decision trees, and any training example set generated by any method can be used.
[0067] (Configuration of information processing device 4) 7 is a block diagram showing an example of the configuration of the information processing device 4 according to this exemplary embodiment. As shown in the figure, the information processing device 4 includes a control unit 40 that controls each unit of the information processing device 4, and a storage unit 41 that stores various data used by the information processing device 4. The information processing device 4 also includes an input unit 33 and an output unit 34.
[0068] The control unit 40 includes a reception unit 401, a decision rule set generation unit 402, a prediction unit 403, a list determination unit 404, and an input data acquisition unit 405. The storage unit 41 stores a decision tree set 411, a decision rule set 412, a training example set 413, and a decision list 414. The input data acquisition unit 405 and the training example set 413 are similar to the elements of the same name in the second exemplary embodiment.
[0069] The reception unit 401 receives a setting of the value of a parameter K. The parameter K indicates the number of decision rules used to calculate a final prediction result. For example, the reception unit 401 may receive the value of K input via the input unit 33 as the setting value of the parameter K.
[0070] The decision rule set generation unit 402 extracts each condition appearing on a path from the root to the leaves of a decision tree included in a decision tree set 411 including at least one decision tree, generates decision rules, and generates a decision rule set including the generated decision rules. In other words, the decision rule set generation unit 402 generates decision rules in which the value of a leaf (end point) of a decision tree is set as an output value y, and each condition appearing on a path from the root to the leaf of the decision tree is set as an input value x. The decision rule set generation unit 402 then generates a decision rule set by performing this process for each leaf (end point) of the decision tree. The decision rule set generation unit 402 also stores the generated decision rule set in the storage unit 41 as a decision rule set 412.
[0071] It should be noted that the decision rule set generation unit 402 is not an essential component of the information processing device 4. The decision rule set generation unit 402 may be omitted. In this case, the information processing device 4 determines the decision list to be output using the decision rule set 311 stored in advance, as in the second exemplary embodiment.
[0072] The prediction unit 403 calculates a prediction result using the predicted values of the top K (K is a natural number greater than or equal to 2) decision rules among the decision rules included in a decision list consisting of multiple decision rules extracted from the decision rule set 412, whose conditions are satisfied by the training examples included in the training example set 413.
[0073] The list determination unit 404 determines a decision list to be output based on the prediction results calculated for each training example included in the training example set 413, for each of the multiple decision lists generated from the decision rule set 412.
[0074] As described above, the information processing device 4 includes a receiving unit 401 that receives the setting of the value of the parameter K indicating the number of decision rules to be used in calculating the final prediction result, and the prediction unit 403 calculates the prediction result using the value of K received by the receiving unit 401.
[0075] According to the above configuration, in addition to the effects of the information processing device 1 according to the first exemplary embodiment, the user can set a desired value for K to determine a decision list suitable for calculating a prediction result using the value of K. This allows the user to set K to a large value when he / she wants to prioritize prediction performance, and to set K to a small value when he / she wants to prioritize the explainability of the prediction result. In other words, according to the above configuration, the user can freely select the trade-off between prediction performance and explainability.
[0076] In this exemplary embodiment, it is assumed that K is set to a value of 2 or more, but it is also possible to set K to 1. Also, in the above-described exemplary embodiment 2, the reception unit 401 may be employed to receive the setting of the value of K.
[0077] As described above, the information processing device 4 is provided with a decision rule set generation unit 402 that extracts each condition that appears on a path from the root to the leaves of a decision tree included in a decision tree set 411 that includes at least one decision tree, generates decision rules, and generates a decision rule set 412 that includes the generated decision rules.
[0078] According to the above configuration, in addition to the effects achieved by the information processing device 1 according to the first exemplary embodiment, an effect is obtained in that a decision rule set based on a decision tree can be automatically generated.
[0079] The set of decision trees may be a set of decision trees used in a random forest. A random forest is a method of generating a set of decision trees from training examples, making predictions using each decision tree included in the set, and synthesizing the prediction results of each decision tree to obtain a final prediction result. Therefore, by generating a decision rule set from the set of decision trees used in the random forest and using a prediction list generated from this decision rule set, predictions can be made using a method similar to that of a random forest. This makes it possible to achieve high prediction performance like that of a random forest.
[0080] (Decision list optimization problem) The prediction unit 403 and the list determination unit 404 determine a decision list to be output by solving an optimization problem of the decision list. As described in the overview, the optimization problem solved by the prediction unit 403 and the list determination unit 404 is an ILP. The following describes a method for making the optimization problem of the decision list an ILP.
[0081] The final prediction result is the decision list L using the prediction values of the top K decision rules that satisfy the conditions. K The optimization problem is to find a decision list L that minimizes the following objective function: K The problem can be defined as finding a solution to the problem. The normalization parameter is λ (a real number). Also, the decision list L K consists of the decision rules contained in the decision rule set R.
[0082] f opt_k =l err (L k ,T)+λ|L k | A training example can be represented by a pair (x, y) of input x (x is a real number) and output y. Thus, a training example set T consisting of m training examples can be expressed as follows:
[0083]
number
[0084] Here, l err (L k ,T) is a decision list L on a training example set T. k is the error function for prediction using λ|L k | is a large size decision list L k is a regularization term that penalizes
[0085] For regression problems, l err (L k For example, the mean squared error (MSE), which is one of the representative error functions, can be used as the error function for the KL divergence (KL divergence) between the true value and the predicted value output by the decision list, and the sum of the KL divergence over the entire training examples can be used as the error function for the KL divergence. The KL divergence is also called information gain.
[0086] Decision List L K can be defined as follows:
number
number
[0087] Decision List L K When making a prediction using K In order from the highest ranking decision rule in l=p→q∈L K We look at the top K decision rules for which x satisfies condition p, and calculate the average of the consequents q of each of them as the predicted value L K For 1≦k≦K, the decision rule l for which x satisfies condition p is the kth in the list order is output as the decision list L for x. K This is called the kth decision rule above.
[0088] Decision list L after optimization K The default rules in are given in advance, and the rule set R = {r1, ..., r |R|}K decision rules in r |R|-K+1 ,…,r |R| corresponds to the default rule.
[0089] Decision list L after optimization K will output the following:
[0090]
number
[0091] Decision List L K A rule in u The height (rank) of is defined as |R|-u+1. Also, R and the decision list L K The relationship with the decision rule r contained in is determined by using the permutation vector π described later.u =l |R|-πu+1 This is expressed as:
[0092] Here, in order to perform ILP conversion, the following variables are introduced.
[0093] A: m×|R| binary matrix. Element A of the matrix iu satisfies the following. That is, observation x (i) is the decision rule r u When the condition A is satisfied, iu is 1, otherwise it is 0.
[0094]
number
[0095]
number
[0096] H: an integer matrix of size m × K. Elements H ik x (i) The decision list L of the kth decision rule for K Indicates the height (ranking) in the
[0097] π: an integer vector of size |R|, with elements π u ∈{1,…,|R|} and decision rule r u Decision list L K Indicates the height (ranking) in the
[0098] δ: |R|×|R| binary matrix. δ uj When = 1, decision rule r u Decision list L K This represents that the height (rank) in is j.
[0099] By using the above variables, the decision list L K The optimization problem can be formulated in ILP as follows:
number
number
number
number
number
number
number
[0100]
number
number
[0101] In addition, the second term of the formula (1) is the above-mentioned objective function: f opt_k =l err (L k ,T)+λ|L k The second term of | corresponds to the large size of the decision list L k For example, this second term may be a normalization term that imposes a penalty on the number of decision rules included in the decision list that is greater, or may be a normalization term that imposes a penalty on the number of conditions included in the decision rules included in the decision list that is greater.
[0102] The above formulas (2) to (6) represent constraints during optimization. Specifically, formulas (2) and (3) express the constraints that, when a rule is the kth decision rule for a certain example, the rule is the kth, ..., Kth decision rule in the decision list L K This indicates that the priority is higher within the
[0103] In addition, formula (4) shows that when a certain decision rule is the k-th decision rule for a certain example, the rule is added to the decision list L by the 1, ..., k-1th decision rules. K Therefore, by using formulas (2) to (4), it is possible to express the condition in which a certain decision rule is the k-th decision rule for a certain example.
[0104] Formula (5) guarantees that out of K decision rules that satisfy the conditions of a certain example, there is one that becomes the kth decision rule. Formula (6) also guarantees that the default rule is included in the decision list L K It is guaranteed that there are K consecutive entries in the
[0105] Equation (7) is a constraint that gives the relationship between π and δ. Equation (8) expresses the relationship between each rule and the decision list L K It ensures that no more than one of
[0106] The above-mentioned calculation method differs from the technique of Non-Patent Document 1 in that the variable D is a tensor with an additional dimension to represent K, and the variable H is a matrix with an additional dimension to represent K. In addition, as the variables D and H are changed as described above, the constraint equations are also different from those of the technique of Non-Patent Document 1. Non-Patent Document 1 does not describe or suggest such an extension, and it is not self-evident that the configuration of this exemplary embodiment can be achieved from Non-Patent Document 1.
[0107] (How to determine which decision list to output) The prediction unit 403 and the list determination unit 404 use the above formulas (2) to (8) to calculate the variable, A j,u , D j,u,k , M j,u , H i,k , π u , and δ u,j These variables indicate the position of a decision rule included in the decision rule set in the decision list. The predetermined condition is a condition for determining whether or not to end the optimization, and is determined in advance.
[0108] Specifically, first, the list determination unit 404 sets each of the above-mentioned variables to an initial value. Then, the prediction unit 403 calculates the value of the objective function using a decision list represented by each of these variables. If the calculated value does not satisfy a predetermined condition, the list determination unit 404 updates each of the above-mentioned variables. The prediction unit 403 and the list determination unit 404 repeat updating each variable and calculating the value of the objective function until the above-mentioned predetermined condition is satisfied. This identifies the value of each variable that indicates the optimal decision list.
[0109] In this way, the prediction unit 403 calculates the value of the objective function (Formula (1)) including an error term (the first term of Formula (1)) indicating the error of the prediction result, using a decision list expressed using variables indicating which decision rule included in the decision rule set is located at which position in the decision list. Furthermore, the list determination unit 404 determines the decision list to be output by repeating the process of updating the variables based on the calculated value of the objective function until the value of the objective function satisfies a predetermined condition.
[0110] According to the above configuration, in addition to the effects of the information processing device 1 according to the first exemplary embodiment, an effect is obtained in that a decision list to be output can be determined by an optimization calculation using an objective function.
[0111] Also, as in the above example, the objective function may be a linear function, and the optimization constraints may be described as equalities or inequalities of the linear function, thereby making it possible to treat the problem of determining the optimal decision list as an ILP, and to efficiently determine the decision list to be output using an optimization solver.
[0112] As described above, the prediction unit 403 calculates the value of the objective function including the constraint term ((the second term of the formula (1))) regarding the number of decision rules included in the decision list. This constraint term may be a constraint term regarding the number of conditions included in the decision rules included in the decision list.
[0113] According to the above configuration, an objective function including a constraint term related to the number of decision rules included in the decision list or the number of conditions included in the decision rules included in the decision list is used. In addition to the effects of the information processing device 1 according to the exemplary embodiment 1, an effect is obtained in that a decision list can be determined with the number of decision rules included in the decision list or the number of conditions included in the decision rules included in the decision list as a constraint. For example, it is possible to determine a decision list with a small number of decision rules or a small number of conditions, that is, a decision list that is composed of simple decision rules and has high interpretability for a user.
[0114] As described above, the variables introduced between the training examples included in the training example set 413 and the decision rules included in the decision rule set 412 include a variable D j,u,k and H i,k Includes:
[0115] According to the above configuration, the first to Kth decision rules for which each training example satisfies the condition, that is, the K decision rules used to calculate the predicted value of each training example, are assigned to the variable D j,u,k and H i,k Thus, the prediction results and errors of each training example can be expressed by these variables, and the value of the objective function can also be expressed. Then, the values of the variables that optimize the decision list can be obtained. Therefore, according to the above configuration, in addition to the effects of the information processing device 1 according to the exemplary embodiment 1, the effect of being able to determine the decision list to be output by the optimization calculation using the objective function can be obtained.
[0116] (Learning method flow) The flow of the learning method executed by the information processing device 4 will be described with reference to Fig. 8. Fig. 8 is a flow diagram showing the flow of the learning method executed by the information processing device 4.
[0117] In S41, the decision rule set generation unit 402 generates a decision rule set from the decision tree set 411. Then, the decision rule set generation unit 402 stores the generated decision rule set in the storage unit 41 as a decision rule set 412.
[0118] As described above, the decision tree set 411 may be generated by a random forest. In this case, the information processing device 4 may perform a process of generating a decision tree set by a random forest prior to S41.
[0119] In S42, the reception unit 401 receives the setting of the value of the parameter K. The user of the information processing device 4 can input a desired value of the parameter K, for example, via the input unit 33. Then, the reception unit 401 sets the value thus input as the value of the parameter K.
[0120] In S43, the list determination unit 404 sets various variables to initial values. Specifically, the list determination unit 404 sets the above-mentioned six variables, i.e., A j,u , D j,u,k , M j,u , H i,k , π u , and δ u,j Set the value of to the initial value.
[0121] In S44, the prediction unit 403 uses each variable set to an initial value in S43 to calculate a prediction result for each training example included in the training example set 413. The prediction result is calculated using the top K prediction values that satisfy the conditions of the training example, among the multiple decision rules included in the decision list expressed using the above variables.
[0122] In S45, the list determination unit 404 calculates the value of the objective function using the prediction result calculated in S44. Specifically, the list determination unit 404 calculates the value of the above-mentioned mathematical formula (1), which is the objective function.
[0123] In S46, the list determination unit 404 judges whether or not the calculation result in S45 satisfies a predetermined condition. If the result of the judgment in S46 is YES, the process proceeds to S48. On the other hand, if the result of the judgment in S46 is NO, the process proceeds to S47.
[0124] In S47, the list determination unit 404 updates the values of the six variables described above based on the value of the objective function calculated in S45. The update may be performed in such a way that the value of the objective function can change in a direction that satisfies a predetermined condition. After this, the process returns to S44.
[0125] In S48, the list determination unit 404 determines the decision list specified by the values of the six variables when it is determined in S46 that the condition is satisfied as the decision list to be output. This makes it possible to output a decision list that is composed of simple rules and has high prediction performance. The list determination unit 404 then stores the determined decision list in the storage unit 41 as the decision list 414, and the process of FIG. 8 is completed.
[0126] In the above process, the variables are updated in S47, and the decision list specified by these variables is updated. Then, a prediction result is calculated for the updated decision list in S44. Therefore, in S48, the decision list to be output is determined based on the prediction result calculated for each training example included in the training example set for each of the multiple decision lists generated from the decision rule set. In addition, the above process (especially S43 to S48) can also be executed by an optimization solver.
[0127] (Prediction method flow) The prediction method executed by the information processing device 4 is the same as the prediction method shown in Fig. 2. Specifically, first, the input data acquisition unit 405 acquires input data to be predicted (S21). Next, the prediction unit 403 calculates prediction values of the top K decision rules whose conditions are satisfied by the input data acquired in S21 among the decision rules included in the decision list 414, and calculates a prediction result using these prediction values.
[0128] Exemplary embodiment 4 (Configuration of information processing device 5) 9 is a block diagram showing an example of the configuration of an information processing device 5 according to this exemplary embodiment. As shown in the figure, the information processing device 5 includes a control unit 50 that controls each unit of the information processing device 5, and a storage unit 51 that stores various data used by the information processing device 5. The information processing device 5 also includes an input unit 33 and an output unit 34.
[0129] The control unit 50 includes a receiving unit 501, a ranking setting unit 502, a prediction unit 503, a list determination unit 504, and an input data acquisition unit 505. The storage unit 51 stores a decision rule set 512, a training example set 513, and a decision list 514. The receiving unit 501, the input data acquisition unit 505, the decision rule set 512, and the training example set 513 are similar to the elements of the same names in the third exemplary embodiment.
[0130] The ranking unit 502 ranks the decision rules included in the decision rule set 512. The ranking method will be described later.
[0131] The prediction unit 503 calculates a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules whose conditions are satisfied by the training examples included in the training example set 513, among the decision rules included in the decision list consisting of multiple decision rules extracted from the decision rule set 512. When calculating this prediction result, the prediction unit 503 calculates the prediction result using the prediction values of the top K decision rules whose ranks are set by the rank setting unit 502.
[0132] The list determination unit 504 determines a decision list to be output based on the prediction result calculated for each training example included in the training example set 513 for each of the multiple decision lists generated from the decision rule set 512. The method of calculating the prediction result by the prediction unit 503 and the method of determining the decision list by the list determination unit 504 will be described in detail later.
[0133] As described above, the information processing device 5 includes the ranking unit 502 that ranks the decision rules included in the decision rule set, and the prediction unit 503 calculates a prediction result using the top K predicted values.
[0134] According to the above configuration, the decision rules are ranked and the prediction result is calculated using the top K predicted values. This makes it unnecessary to consider the order of the decision rules in the decision list when determining the decision list to be output.
[0135] For example, for a decision list including three decision rules A to C, taking into consideration the order of the decision rules, one must be selected from six options: ABC, ACB, BAC, BCA, CAB, and CBA.
[0136] On the other hand, if the decision rules A to C are ranked, the one to be output can be determined according to the ranking. For example, if the rules are ranked in the order of A, B, C, the decision rules to be included in the decision list to be output can be arranged in the order of A, B, C.
[0137] Thus, according to the above configuration, in addition to the effect of the information processing device 1 of exemplary embodiment 1, the effect of being able to complete the process of determining the decision list to be output in a shorter time than when the order of arrangement is taken into consideration is obtained.
[0138] (Example of ranking) As described above, in prediction using a decision list, the decision rules are checked in order of ranking to find the top K decision rules that satisfy the conditions, and the final prediction result is calculated from the predicted values of those decision rules.
[0139] For this reason, it is preferable to rank a more general decision rule that applies to more examples lower in the decision list, and rank a more specific decision rule that applies to only a smaller number of examples higher in the decision list.
[0140] Therefore, the ranking unit 502 may, for example, count the number of training examples that satisfy the conditions of each decision rule included in the decision rule set 512, and rank the decision rules in ascending order of the number of training examples that satisfy the conditions of the decision rule.
[0141] Furthermore, in the decision list, it is desirable that decision rules with more certain prediction results be ranked higher than decision rules with more ambiguous prediction results.
[0142] Therefore, when setting the order of decision rules predicting a solution to a regression problem, the order setting unit 502 may calculate the standard deviation of the predicted value (output y) of a training example that satisfies the condition of each decision rule included in the decision rule set 512. Then, the order setting unit 502 may rank the decision rules in ascending order of the calculated standard deviation.
[0143] In addition, when setting a rank for a decision rule that predicts a solution to a classification problem, the rank setting unit 502 may perform the ranking based on the difference between a predicted value for a training example that satisfies the conditions of the decision rule and a predicted value of a comparison target.
[0144] The predicted value to be compared may be, for example, the predicted value of the default rule described above. In this case, the ranking setting unit 502 uses the prediction of the default rule as a standard and ranks the decision rules in order of how well the prediction is narrowed down compared to the prediction of the default rule.
[0145] As an index for evaluating whether the predictions have been successfully narrowed down, for example, the KL divergence can be used. When ranking is performed using the KL divergence, the ranking setting unit 502 calculates the KL divergence for the predicted value of the default rule and the predicted value of each decision rule included in the decision rule set 512, and ranks the decision rules in descending order of the value of the KL divergence.
[0146] In this way, the ranking unit 502 may rank the decision rules based on the difference between the predicted value for the training example that satisfies the condition of the decision rule and the predicted value of the comparison target. According to this configuration, in addition to the effect of the information processing device 1 according to the exemplary embodiment 1, the effect of being able to rank the decision rules in order of the possibility of calculating a more appropriate predicted value can be obtained. In this case, the optimization calculation using the objective function includes heuristic elements such as the KL divergence, so that the optimization is approximate.
[0147] (Decision list optimization problem) As described above, the information processing device 5 includes the ranking setting unit 502. Therefore, the list determination unit 504 does not need to consider rearrangement of the decision rules, and only needs to determine whether or not to include each decision rule in the decision list. Therefore, in this exemplary embodiment, the optimization problem of the decision list is simplified compared to the exemplary embodiment 3.
[0148] Specifically, instead of π used in the third exemplary embodiment, we introduce a binary vector γ of size |R|. u When is 1, decision rule r u is the decision list L K Therefore, prepare an initialized empty decision list, and for all 1≦u≦|R|, sequentially select γ u Check γ u Decision rule r if and only if =1 u Decide on the list L K By adding the list to the end of the list, the optimized decision list L K can be obtained.
[0149] Accordingly, the objective function of formula (1) is changed to the following formula (9). Compared with formula (1), formula (2) uses a decision list L with a larger size. k The second term, which is a regularization term that penalizes for , has changed. Note that the second term is also a constraint term.
[0150]
number
number
[0151] Furthermore, the equations (2) to (4) and (6) expressing the constraint conditions are changed as follows:
number
number
number
[0152]
number
[0153] In the optimization calculation using the above formulas, instead of π, which is an integer vector of size |R|, a binary vector γ of size |R| is used, so the search space is narrower than that of the example of exemplary embodiment 3 using π. Also, while exemplary embodiment 3 required formulas (7) and (8) to express π, these formulas are not required in this exemplary embodiment, and it is possible to realize the ILP expression using only formulas (5) and (10) to (13).
[0154] (Learning method flow) The learning method executed by the information processing device 5 is generally similar to the learning method shown in Fig. 8. The main differences are that the process of S41 is not performed, π and δ are not included in the variables to be set in S43 and updated in S47, and the ranking is set by the ranking setting unit 502 at a stage prior to the calculation of the prediction result in S44. In addition, various formulas used to determine the decision list to be output are also different from the learning method described in Fig. 8, such as the objective function in S45 being changed to the above-mentioned formula (9).
[0155] (Prediction method flow) The prediction method executed by the information processing device 5 is the same as the prediction method shown in Fig. 2. Specifically, first, the input data acquisition unit 505 acquires input data to be predicted (S21). Next, the prediction unit 503 calculates prediction values of the top K decision rules whose conditions are satisfied by the input data acquired in S21 among the decision rules included in the decision list 514, and calculates a prediction result using these prediction values.
[0156] [Reference example] In the above-described exemplary embodiments, the parameter K indicating the number of decision rules used to calculate the final prediction result is equal to or greater than 2. However, the method of reducing the time required for the process of determining the decision list to be output by prioritizing each decision rule included in the decision rule set is also effective when the parameter K is 1.
[0157] In this reference example, an information processing device 6 that outputs an optimal determination list when the parameter K is equal to or greater than 1 will be described. Fig. 10 is a block diagram showing the configuration of the information processing device 6 according to this reference example. As shown in the figure, the information processing device 6 includes a ranking setting unit 61, a prediction unit 62, and a list determination unit 63.
[0158] The ranking setting unit 61 ranks the decision rules included in the decision rule set in the same manner as the above-mentioned ranking setting unit 502.
[0159] The prediction unit 62 calculates a prediction result based on the prediction value of one or more decision rules whose conditions are satisfied by the training examples included in the training example set, among the decision rules included in the decision list consisting of multiple decision rules extracted from the decision rule set. In this way, in this reference example, there may be only one decision rule that satisfies the conditions. This is because it is assumed that the parameter K is a value of 1 or more.
[0160] Note that the processing when the parameter K is 2 or more is similar to that in the fourth exemplary embodiment, so the following describes the case where the parameter K is 1. In this case, the prediction unit 62 calculates a prediction result based on the predicted value of the first decision rule (the highest-ranked decision rule among the decision rules that satisfy the condition) among the decision rules included in the decision list consisting of a plurality of decision rules extracted from the decision rule set.
[0161] The list determination unit 63 determines the decision list to be output based on the prediction results calculated for each training example included in the training example set and the ranking set by the ranking setting unit 61, for each of the multiple decision lists generated from the decision rule set.
[0162] As described above, the information processing device 6 includes a prediction unit 62 that calculates a prediction result based on the predicted value of the first decision rule for which a training example included in the training example set satisfies a condition, among the decision rules included in a decision list consisting of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules that combine a condition and a predicted value when the condition is satisfied; a ranking setting unit 61 that ranks each decision rule included in the decision rule set; and a list determination unit 63 that determines the decision list to be output based on the prediction result and the ranking calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set.
[0163] According to the above configuration, the decision rules are ranked, and the prediction result is calculated based on the prediction value of the first decision rule for which the training example satisfies the condition. This makes it unnecessary to consider the order of the decision rules in the decision list when determining the decision list to be output. Therefore, according to the above configuration, it is possible to complete the process of determining the decision list to be output in a shorter time than when the order is taken into consideration.
[0164] The learning method executed by the information processing device 6 is similar to the learning method of the fourth exemplary embodiment, except that K=1.
[0165] The information processing device 6 may also include an input data acquisition unit 21 (see FIG. 1). In this case, the input data acquisition unit 21 acquires input data. Then, the prediction unit 62 calculates a prediction result using a prediction value of the highest decision rule, among the decision rules included in the decision list output by the list determination unit 63, whose condition is satisfied by the input data acquired by the input data acquisition unit 21.
[0166] [Modifications] The execution subject of each process described in each of the above exemplary embodiments and reference examples is arbitrary and is not limited to the above examples. In other words, an information processing system having the same functions as the information processing devices 1 to 6 can be constructed by multiple devices that can communicate with each other.
[0167] [Software implementation example] Some or all of the functions of the information processing devices 1 to 6 may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.
[0168] In the latter case, the information processing devices 1 to 6 are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 11. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing devices 1 to 6. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 1 to 6.
[0169] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.
[0170] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.
[0171] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.
[0172] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.
[0173] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.
[0174] (Appendix 1) An information processing device comprising: a prediction means for calculating a prediction result using the predicted values of the top K (K is a natural number of 2 or more) decision rules whose training examples included in a training example set satisfy the condition, among the decision rules included in a decision list composed of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied; and a list determination means for determining the decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set. With this configuration, it is possible to improve the prediction performance of prediction using the decision list.
[0175] (Appendix 2) The information processing device according to appendix 1, wherein the prediction means calculates a value of an objective function including an error term indicating an error of the prediction result, using the decision list expressed using a variable indicating which decision rule included in the decision rule set is located at which position in the decision list, and the list determination means determines the decision list to be output by repeating a process of updating the variable based on the calculated value of the objective function until the value of the objective function satisfies a predetermined condition. With this configuration, it is possible to determine the decision list to be output by optimization calculation using the objective function.
[0176] (Appendix 3) The information processing device according to Supplementary Note 2, wherein the prediction means calculates a value of the objective function including a constraint term on the number of the decision rules included in the decision list, or a constraint term on the number of the conditions included in the decision rules included in the decision list. With this configuration, it is possible to determine a decision list with the number of decision rules included in the decision list, or the number of conditions included in the decision rules included in the decision list as a constraint.
[0177] (Appendix 4) The information processing device according to claim 2 or 3, wherein the variables include, for each training example included in the training example set, variables representing K decision rules from 1st to Kth in the decision list for which the training example satisfies the condition. With this configuration, the decision list to be output can be determined by optimization calculation using an objective function.
[0178] (Appendix 5) The information processing device according to any one of Supplementary Notes 1 to 4, further comprising a receiving means for receiving a setting of the value of K, and the prediction means calculates the prediction result using the value of K received by the receiving means. According to this configuration, a user can set the value of K to a desired value, thereby causing a determination list suitable for calculating a prediction result using that value of K to be determined.
[0179] (Appendix 6) The information processing device according to any one of appendices 1 to 5, further comprising a decision rule set generating means for generating the decision rules by extracting each condition appearing on a path from a root to a leaf of a decision tree included in a decision tree set including at least one decision tree, and generating the decision rule set including the generated decision rules. With this configuration, a decision rule set based on a decision tree can be automatically generated.
[0180] (Appendix 7) The information processing device according to any one of claims 1 to 3, further comprising a ranking setting means for ranking each of the decision rules included in the decision rule set, and the prediction means calculates the prediction result using the K prediction values with the highest ranking. With this configuration, it is possible to complete the process of determining the decision list to be output in a shorter time than in the case where the sorting order is taken into consideration.
[0181] (Appendix 8) The information processing device according to claim 7, wherein the ranking setting means ranks the decision rules based on a difference between a predicted value for the training example that satisfies a condition of the decision rule and a predicted value of a comparison target. With this configuration, it is possible to rank the decision rules in order of a higher probability of calculating a more appropriate predicted value.
[0182] (Appendix 9) An information processing device comprising: an input data acquisition means for acquiring input data to be predicted; and a prediction means for calculating a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules for which the input data satisfies the conditions, among the decision rules included in a decision list consisting of a plurality of decision rules each combining a condition with a prediction value when the condition is satisfied. With this configuration, it is possible to improve prediction performance compared to a conventional method that uses only the top prediction value in the prediction list.
[0183] (Appendix 10) A learning method comprising: at least one processor calculating a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules whose training examples included in a training example set satisfy the condition, among the decision rules included in a decision list composed of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied, and determining the decision list to be output based on the prediction result calculated for each training example included in the training example set for each of the plurality of decision lists generated from the decision rule set. With this configuration, it is possible to improve the prediction performance of predictions using decision lists.
[0184] (Appendix 11) A learning program that causes a computer to function as: a prediction means for calculating a prediction result using the predicted values of the top K (K is a natural number of 2 or more) decision rules whose training examples included in a training example set satisfy the condition, among the decision rules included in a decision list composed of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied, and a list determination means for determining the decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set. With this configuration, it is possible to improve the prediction performance of prediction using a decision list.
[0185] [Additional Note 3] A part or all of the above-described embodiments can be further expressed as follows: An information processing device including at least one processor, the processor executing a prediction process of calculating a prediction result using predicted values of top K (K is a natural number of 2 or more) decision rules whose training examples included in a training example set satisfy the condition, out of the decision rules included in a decision list composed of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied, and a list determination process of determining the decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the plurality of decision lists generated from the decision rule set.
[0186] An information processing device comprising at least one processor, the processor executing: a data acquisition process for acquiring input data to be predicted; and a prediction process for calculating a prediction result using the predicted values of the top K (K is a natural number equal to or greater than 2) decision rules for which the input data satisfies the conditions, out of a plurality of decision rules included in a decision list that includes a combination of conditions and predicted values when the conditions are satisfied.
[0187] In addition, these information processing devices may further include a memory, and this memory may store a learning program for causing the processor to execute the prediction process and the list determination process, or a prediction program for causing the processor to execute the data acquisition process and the prediction process. In addition, these programs may be recorded in a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]
[0188] 1 Information processing device 2 Information processing device 11 Prediction section 21 Input data acquisition section 12 List determination section 22 Prediction section 3. Information processing equipment 302 Prediction Department 303 List Decision Unit 311 Decision Rule Set 312 Training example set 313 Decision List 4 Information processing device 5 Information processing device 401 Reception Department 501 Reception Department 502 Ranking Division 402 Decision rule set generation unit 403 Prediction Department 503 Prediction Department 404 List determination unit 504 List determination unit 405 Input data acquisition unit 505 Input data acquisition unit 411 Decision tree set 412 Decision rule set 512 Decision rule set 413 Training example set 513 Training example set 414 Decision List 514 Decision List
Claims
1. a prediction means for calculating a prediction result using predicted values of the top K (K is a natural number of 2 or more) decision rules whose training examples in a training example set satisfy the condition, out of the decision rules included in a decision list consisting of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied; and a list determination means for determining the decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the multiple decision lists generated from the decision rule set.
2. the prediction means calculates a value of an objective function including an error term indicating an error of the prediction result, using the decision list expressed using a variable indicating which decision rule included in the decision rule set is located at which position in the decision list; 2 . The information processing apparatus according to claim 1 , wherein the list determination means determines the determination list to be output by repeating a process of updating the variables based on the calculated value of the objective function until the value of the objective function satisfies a predetermined condition.
3. The information processing apparatus according to claim 2 , wherein the prediction means calculates a value of the objective function including a constraint term regarding the number of the decision rules included in the decision list, or a constraint term regarding the number of the conditions included in the decision rules included in the decision list.
4. The information processing device according to claim 2 or 3, wherein the variables include variables representing, for each training example included in the training example set, K decision rules from 1st to Kth in the decision list for which the training example satisfies the condition.
5. A reception means for receiving a setting of the value of K, The information processing apparatus according to claim 1 , wherein the prediction means calculates the prediction result by using the value of K accepted by the acceptance means.
6. 6. The information processing device according to claim 1, further comprising a decision rule set generation means for generating the decision rules by extracting each condition that appears on a path from a root to a leaf of a decision tree included in a decision tree set including at least one decision tree, and generating the decision rule set including the generated decision rules.
7. a ranking setting means for ranking each of the decision rules included in the decision rule set, The information processing apparatus according to claim 1 , wherein the prediction means calculates the prediction result using the K predicted values having the highest ranking.
8. An input data acquisition means for acquiring input data to be predicted, 2. The information processing device according to claim 1, wherein the prediction means calculates a prediction result using prediction values of top K (K is a natural number equal to or greater than 2) decision rules among the decision rules included in the decision list determined by the list determination means, for which the input data satisfies the condition.
9. At least one processor calculating a prediction result using the prediction values of the top K (K is a natural number of 2 or more) decision rules whose training examples in a training example set satisfy the condition, out of the decision rules included in a decision list consisting of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied; and determining the decision list to be output based on the prediction result calculated for each training example included in the training example set, for each of the multiple decision lists generated from the decision rule set.
10. Computer, a prediction means for calculating a prediction result using predicted values of the top K (K is a natural number of 2 or more) decision rules whose training examples included in a training example set satisfy the condition, among the decision rules included in a decision list composed of a plurality of decision rules extracted from a decision rule set, which is a set of decision rules combining a condition and a predicted value when the condition is satisfied; and A learning program that functions as a list determination means that determines the decision list to be output based on the prediction results calculated for each training example included in the training example set, for each of the multiple decision lists generated from the decision rule set.
Citation Information
Patent Citations
Language processor
JP2001229163A
Decision list learning device, decision list learning method, and decision list learning program
WO2020059136A1
