Generation device, generation method, and generation program

Abstract

To search for a branch condition effective for prediction.SOLUTION: The generation device solves a QUBO problem based on a plurality of explanatory variables and objective variables for each sample and a plurality of unit conditions to generate a combination feature. The QUBO problem is a problem of calculating a first variable and a second variable that minimize a value of a second loss function based on an error function for calculating a prediction error of a decision tree, a first loss function for obtaining a sum of the first constraint and the second constraint, and a penalty for a unit condition, the first constraint being a constraint that minimizes the first variable for each sample indicating how many of one or more unit conditions included in a branch condition are satisfied by an explanatory variable for each number of unit conditions satisfied by the explanatory variable, the second constraint is a constraint that minimizes a second variable for each sample indicating whether or not the branch condition is one unit condition for each number of unit conditions satisfied by the explanatory variable, and the penalty is a term that suppresses selection of the branch condition using the second variable.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a generating device, a generating method, and a generating program for generating information. [Background technology]

[0002] Japanese Patent Laid-Open Publication No. 2023-113393 (Patent Document 1) is a background technology in this technical field. This publication states that "the estimation accuracy of the branching conditions of a decision tree is improved. An estimator learning device that learns an estimator that searches for branching conditions of a decision tree that estimates a target variable from data on explanatory variables, wherein the estimator 12 includes a QUBO problem conversion unit that converts a prediction error minimization problem in the search for branching conditions into a QUBO problem or a first problem equivalent to the QUBO problem, a QUBO problem calculation unit that calculates the first problem converted by the QUBO problem conversion unit, and a branching condition generation unit that generates branching conditions based on the calculation results of the QUBO problem calculation unit." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-113393 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology in Patent Document 1 improves the prediction accuracy of decision trees by converting the problem of searching for decision tree branching conditions that minimize estimation errors into an approximately equivalent Quadratic Unconstrained Binary Optimization (QUBO) problem and solving it using an Ising machine, etc. The branching conditions obtained in this way are, for example, the logical product of conditions for multiple explanatory variables, and can be thought of as new features (called combined features) that are highly correlated with the objective variable.

[0005] However, the purpose of combination features is to improve the prediction accuracy of decision trees, and they are not necessarily useful to users as new features. For example, when there are two explanatory variables that are highly correlated with the target variable, the logical product of the conditions for these explanatory variables can also be a combination feature that is highly correlated with the target variable. However, if the user's goal is to find a set of explanatory variables that only become strongly correlated when combined, the combination feature will not fulfill the purpose and will not be useful.

[0006] An object of the present invention is to search for a branch condition that is effective for prediction. [Means for solving the problem]

[0007] A feature generator according to one aspect of the invention disclosed in the present application is a generation device that includes a processor that executes a program and a storage device that stores the program, and generates branching conditions for a decision tree that estimates a target variable from a plurality of explanatory variables, wherein the processor performs an input process of inputting the plurality of explanatory variables and the target variable for each sample in a sample set, and a plurality of simplex conditions, and solves a QUBO problem based on the plurality of explanatory variables and the target variable for each sample input by the input process, and the plurality of simplex conditions, and the QUBO problem is solved by using an error function that calculates a prediction error of the decision tree and a simplex condition that is satisfied by the explanatory variables. a loss function for calculating the sum of a first constraint for each sample that indicates how many of one or more simplex conditions included in a branching condition the explanatory variable satisfies for each number of simplex conditions; a second constraint for each sample that indicates whether the branching condition is one simplex condition or not for each number of simplex conditions satisfied by the explanatory variable; and a penalty for the simplex condition that uses the second variable to suppress selection for the branching condition; and an output process that outputs the combined features generated by the generation process. [Effects of the Invention]

[0008] According to the exemplary embodiment of the present invention, it is possible to search for a branch condition that is effective for prediction. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiment. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating an example of the hardware configuration of a feature generation device. [Figure 2] FIG. 2 is a block diagram of an example of a functional configuration of the feature generating device according to the first embodiment. [Figure 3] FIG. 3 is a block diagram illustrating an example of the functional configuration of the QUBO problem conversion unit. [Figure 4] FIG. 4 is an explanatory diagram showing an example of a decision tree that reduces prediction errors. [Figure 5] FIG. 5 is a diagram illustrating an example of a data structure of the explanatory variable DB according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a data structure of the objective variable DB according to the first embodiment. [Figure 7] FIG. 7 is an explanatory diagram showing an example of the data structure of the search parameter DB. [Figure 8] FIG. 8 is a diagram illustrating an example of a data structure of the unit condition DB according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a data structure of the penalty DB according to the first embodiment. [Figure 10] FIG. 10 is a flowchart of an example of a procedure for generating features according to the first embodiment. [Figure 11] FIG. 11 is a block diagram of an example of a functional configuration of a feature generating device according to the second embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of a data structure of the explanatory variable DB according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a data structure of the objective variable DB according to the second embodiment. [Figure 14]FIG. 14 is a diagram illustrating an example of a data structure of the unit condition DB according to the second embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a data structure of the penalty DB according to the second embodiment. [Figure 16] FIG. 16 is an explanatory diagram illustrating an example of the data structure of the condition selection ratio DB. [Figure 17] FIG. 17 is an explanatory diagram illustrating an example of the data structure of the branching condition DB. [Figure 18] FIG. 18 is a flowchart of an example of a procedure for generating features according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION [Example]

[0010] In Example 1, an example will be described in which important combination features are discovered in a task of estimating sales of ice cream (a general term for ice cream, ice milk, lacto ice cream, and frozen desserts). Here, a combination feature refers to a condition in which multiple explanatory variables are combined using logical product or logical sum, such as "temperature is 20°C or higher" and "type of ice cream is frozen dessert." However, it is desirable that the combination feature discovered here is one that does not have a strong correlation with sales as a single explanatory variable, but becomes strongly correlated when combined. Note that whether logical product or logical sum is used to combine multiple explanatory variables is set in advance.

[0011] <Figure 1 Example of hardware configuration of feature generation device> FIG. 1 is a block diagram showing an example of the hardware configuration of a feature generation device. The feature generation device 100A includes a processor 101, a storage device 102, an input device 103, an output device 104, and a communication interface (communication IF) 105. The processor 101, the storage device 102, the input device 103, the output device 104, and the communication IF 105 are connected via a bus 106. The processor 101 controls the feature generation device 100A. The storage device 102 serves as a working area for the processor 101. The storage device 102 is a non-transitory or temporary recording medium that stores various programs and data. Examples of the storage device 102 include a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory. The input device 103 inputs data. Examples of the input device 103 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 104 outputs data. Examples of the output device 104 include a display, a printer, and a speaker. The communication IF 105 connects to a network and transmits and receives data.

[0012] <Figure 2: Example of functional configuration of feature generation device 100A> 2 is a block diagram illustrating an example of a functional configuration of a feature generation device 100A according to Example 1. The feature generation device 100A includes an interface 201, a DB (database) 202, and a branch condition searcher 203 as functions.

[0013] The interface 201 includes an input unit 211 and an output unit 212. The input unit 211 receives as input data including explanatory variables and target variables, search parameters, and simplex conditions.

[0014] The explanatory variables are stored in an explanatory variable DB 221, the objective variables are stored in an objective variable DB 222, the search parameters are stored in a search parameter DB 223, and the simplex conditions are stored in a simplex condition DB 224. Here, the simplex condition represents a condition that one explanatory variable should satisfy, such as "temperature ≧ 20".

[0015] The DB 202 includes an explanatory variable DB 221 (FIG. 4), a response variable DB 222 (FIG. 5), a search parameter DB 223 (FIG. 6), a simplex condition DB 224 (FIG. 7), and a penalty DB 225 (FIG. 8).

[0016] The branching condition searcher 203 includes a conversion unit 231, a QUBO problem calculation unit 232, and a branching condition generation unit 233.

[0017] The conversion unit 231 converts the problem of searching for a branching condition that reduces the prediction error of the decision tree into a QUBO problem. Details of the conversion unit 231 will be described later with reference to FIG.

[0018] The QUBO problem calculation unit 232 performs calculations to solve the QUBO problem generated by the conversion unit 231. The QUBO problem calculation unit 232 may be, for example, any of simulated annealing implemented on the processor 101 (CPU (Central Processing Unit) or GPU (Graphics Processing Unit)), a quantum annealing machine that uses the properties of quantum mechanics, a coherent Ising machine that uses the properties of light, and a digital annealer configured with a digital circuit that uses a CMOS (Complementary Metal Oxide Semiconductor) or an FPGA (Field Programmable Gate Array).

[0019] The branching condition generation unit 233 generates a branching condition such as "temperature ≧ 20" and "type of ice cream = frozen dessert" from the calculation result of the QUBO problem calculation unit 232. The branching condition generated here is displayed on the output unit 212 as a combined feature.

[0020] <Figure 3 Conversion unit 231> 3 is a block diagram showing an example of the functional configuration of the conversion unit 231. The conversion unit 231 includes a first conversion unit 301 and a second conversion unit 302. The conversion unit 231 uses a decision tree that reduces prediction errors.

[0021] [Figure 4 Decision tree] 4 is an explanatory diagram showing an example of a decision tree that reduces prediction errors. The conversion unit 231 samples all or some of the samples s stored in the explanatory variable DB 221 and the objective variable DB 222 to create a sample set S, and divides the sample set S into a subset S1 that satisfies the branching condition and a subset S0 that does not. The conversion unit 231 calculates predicted values ​​y0 and y1 of the objective variables of each set S0 and S1 as the average values ​​of the objective variables of the subsets S0 and S1, respectively.

[0022] For example, the predicted value y1 of the objective variable is the average value of the objective variable of sample IDs 401: 3, 5, 12, ... (satisfies the branching condition), and the predicted value y0 of the objective variable is the average value of the objective variable of sample IDs 401: 2, 6, 9, ... (does not satisfy the branching condition).

[0023] The first conversion unit 301 is a module that converts the problem of searching for a branching condition that minimizes the prediction error of the decision tree 400 for the sample set S into a QUBO problem, and is implemented, for example, by the method described below.

[0024] If the branching condition is the logical product of some of the simplex conditions C1, C2, ... stored in the simplex condition DB 224, the branching condition can be expressed by a binary variable ξi that takes the value "1" when the simplex condition Ci (i is any natural number) is included in the branching condition, and "0" otherwise. The binary variable ξi is the calculation result of the QUBO problem calculation unit 232.

[0025] Considering the mean squared error (MSE) as the prediction error of the decision tree 400, it is expressed by the following formula (1).

[0026]

number

[0027] In the above formula (1), the first term on the right side represents the residual sum of squares of the predicted value y0 in the subset S0 that does not satisfy the branching condition. Similarly, the second term on the right side represents the residual sum of squares of the predicted value y1 in the subset S1 that satisfies the branching condition. In the above formula (1), y s is the objective variable of sample s in sample set S. θ m s is a binary variable that takes "1" if the explanatory variable of sample s does not satisfy m simplex conditions included in the branching conditions, and takes "0" otherwise. Also, m is an integer value equal to or greater than 0. In the above formula (1), m is fixed at 0, but in the following formula (2), the sum of all m is taken. Also, at this time, θ m s must satisfy the following constraints (2) and (3).

[0028]

number

[0029] The above formula (2) is the above-mentioned θ m s The above formula (3) defines θ m s This shows the constraint that θ must be a one-hot vector for m. m s This shows the constraint that x must be 1 for only one m and 0 for the other m. i s is a binary variable that takes the value "1" when sample s satisfies the simplex condition Ci and "0" when it does not.

[0030] The search for a branching condition that reduces the prediction error is performed by finding a θ that minimizes the prediction error (MSE) of the decision tree 400, which is the right-hand side of the above formula (1), while satisfying the above formula (2) and the above formula (3). m s and ξi.

[0031] Here, y0 and y1 are expressed by the following equations (4) and (5), so the above equation (1) is θ m s is not a quadratic function.

[0032]

number

[0033] Therefore, for example, by defining the Square-Weighted MSE (SWMSE) by multiplying the first and second terms on the right side of the above equation (1) by |S0| / |S| and |S1| / |S|, respectively, the SWMSE can be calculated as follows: m s This reduces to a quadratic formula for

[0034] Furthermore, the constraints of the above formulas (2) and (3) are both satisfied when the square of the difference D between the left and right sides is at its minimum value of 0. Therefore, for example, satisfying the constraints of the above formulas (2) and (3) can be converted into a problem of minimizing λD, which is the square of the difference D between the left and right sides multiplied by an appropriate positive number λ. In this case, λD is θD for both the above formulas (2) and (3). m s and a quadratic expression in ξi.

[0035] Therefore, if the sum of SWMSE and λD for all the above constraints is the loss function H0, then the θ that minimizes the loss function H0 is m s The problem of finding ξi is a QUBO problem.

[0036] The above implementation is just one example. For example, instead of reducing it to a QUBO problem by considering SWMSE instead of MSE, it is also possible to expand y0 and y1 with binary variables, then perform variable substitution on the cubic terms of the binary variables to reduce them to quadratic expressions, and add constraints due to the variable substitution to reduce it to a QUBO problem.

[0037] Furthermore, for example, a term may be added that restricts the number of ξi that are "1" to not exceed the maximum number of simplex conditions that can be selected by setting an upper limit on the number of simplex conditions included in the branching conditions as the maximum number of simplex conditions that can be selected.

[0038] The second conversion unit 302 generates a QUBO problem with a loss function H=H0+H1, which is the loss function H0 generated by the first conversion unit 301 plus a penalty term H1 to prevent undesirable branching conditions from being obtained as a solution.

[0039] The penalty term H1 is given so that when the simplex condition Ci is included in the branching conditions, the loss function H becomes extra large by the penalty wi, and the larger the penalty wi, the less likely the simplex condition Ci is to be selected as a branching condition. This is realized by giving the penalty term H1 using the following formula (6).

[0040]

number

[0041] The penalty wi associated with the simplex condition Ci is calculated by the method described below and stored in the penalty DB 225. However, the above formula (6) is only an example, and for example, when two simplex conditions Ci and Cj are selected simultaneously, wij × ξi × ξj may be added to the penalty term H1 so that the loss function H increases by wij, or a higher-order term of ξi may be added so that the loss function H increases when three or more simplex conditions are selected simultaneously. When a cubic or higher-order term of ξi appears in the penalty term H1, the problem is reduced to a QUBO problem by introducing an auxiliary binary variable and an additional constraint equation.

[0042] In the first embodiment, for example, the frequency of occurrence of a combined feature that includes an explanatory variable such as "presence or absence of TV commercials," which has a strong correlation with ice cream sales, is reduced. This is achieved by setting a larger penalty for a single condition that reduces the prediction error when used alone as a branching condition of the decision tree 400. If the prediction error of the decision tree 400 when the sample set S is divided based on the single condition Ci is Li, the penalty wi is given by, for example, the following formula (7):

[0043]

number

[0044] where L S is the prediction error when the sample set S is not divided, and for example, when MSE is used as the prediction error, it corresponds to the variance of the objective variable of the sample set S. The positive parameter β is creativity, and is input to the input unit 211 and stored in the search parameter DB 223. However, the above formula (7) is only an example, and for example, the penalty w 0i may be added to the right side of equation (7).

[0045] <Figure 5 Explanatory variable DB221> 5 is an explanatory diagram illustrating an example of a data structure of the explanatory variable DB 221 according to the first embodiment. The explanatory variable DB 221 is a table that stores data of explanatory variables of each sample s, and stores a sample ID 401 and an explanatory variable associated with each sample ID 401 as column values. The explanatory variables may be in any data format as long as they correspond to the sample ID 401, such as continuous numerical values ​​such as temperature 502 and humidity 503, class information such as type of ice cream 504, or a binary variable 505 indicating the presence or absence of a television commercial. In addition, some explanatory variables may be missing.

[0046] <Fig. 6 Objective variable DB222> 6 is an explanatory diagram illustrating an example of the data structure of the objective variable DB 222 according to the first embodiment. The objective variable DB 222 is a table that stores data of the objective variables of each sample s, and stores, as column values, a sample ID 401 and an objective variable associated with each sample ID 401. The objective variable is given as a real value, such as ice cream sales 602, for example.

[0047] <Figure 7 Search parameter DB223> 7 is an explanatory diagram showing an example of the data structure of the search parameter DB 223. The search parameter DB 223 is a table that stores parameters used when the conversion unit 231 converts into QUBO problems, and stores search parameters 701 and values ​​702 associated with each search parameter 701 as column values. Examples of the search parameters 701 include creativity (a positive parameter β) and the maximum number of selected simple conditions. The maximum number of selected simple conditions is the maximum number of simple conditions included in the selectable combination feature. The number of searches and the loop counter are used in Example 2.

[0048] <Figure 8 Single unit condition DB224> 8 is an explanatory diagram illustrating an example of a data structure of the simplex condition DB 224 according to the embodiment 1. The simplex condition DB 224 is a table that stores simplex conditions, which are elements that configure branch conditions by logical operations, and stores IDs 801 and simplex conditions 802 associated with each ID 801 as column values.

[0049] <Figure 9 Penalty DB225> 9 is an explanatory diagram illustrating an example of the data structure of the penalty DB 225 according to the embodiment 1. The penalty DB 225 is a table that stores penalties wi used when converting to QUBO problems in the second conversion unit 302, and stores, as column values, simplex conditions 802 and penalties 902 associated with each simplex condition 802.

[0050] <Figure 10 Feature generation process> FIG. 10 is a flowchart of an example of a procedure for generating features according to the first embodiment.

[0051] (Step S1001) The feature generation device 100A receives input of data including explanatory variables and response variables via the input unit 211, and proceeds to step S1002. The explanatory variables are stored in the explanatory variable DB 221, and the response variables are stored in the response variable DB 222.

[0052] (Step S1002) The feature generation device 100A receives input of search parameters and simplex conditions via the input unit 211, and proceeds to step S1003. The search parameters are stored in the search parameter DB 223, and the simplex conditions are stored in the simplex condition DB 224.

[0053] (Step S1003) The feature generation device 100A converts the problem of searching for a branching condition that reduces the prediction error into a QUBO problem using the conversion unit 231, and proceeds to step S1004.

[0054] (Step S1004) In feature generation device 100A, QUBO problem calculation unit 232 calculates the QUBO problem converted by conversion unit 231, branch condition generation unit 233 generates a branch condition from the calculation result ξi, and proceeds to step S1005. Specifically, for example, when three simple conditions C1: "Temperature > 30°C", C2: "Humidity > 50%", and C3: "The type of ice cream is frozen" obtained from simple condition DB 224 are given, and as a result of solving the QUBO problem, ξ1 = 1, ξ2 = 0, and ξ3 = 1 are obtained, the generated branch condition is "C1 and C3" (i.e., "Temperature > 30°C and the type of ice cream is frozen").

[0055] In addition, θ m s is also one of the calculation results by the QUBO problem calculation unit 232, but as described above, θ m sis a variable that indicates how many of the simplex conditions Ci given by the binary variable ξi each sample s does not satisfy. Therefore, in the above example, if sample s does not satisfy 0 of the simplex conditions C1 and C3, then θ0 s =1,θ1 s =0,θ2 s = 0. If sample s does not satisfy one of the simplex conditions C1 and C3, then θ0 s =0,θ1 s =1,θ2 s = 0. If sample s does not satisfy both of the simplex conditions C1 and C3, then θ0 s =0,θ1 s =0,θ2 s =1.

[0056] (Step S1005) The feature generation device 100A outputs the branching conditions as combined features via the output unit 212.

[0057] As described above, according to the first embodiment, by searching for a new feature that combines a plurality of individual conditions (features) as a branching condition, it is possible to reduce the prediction error of the decision tree 400. For example, when predicting "sales of a certain ice cream on a certain day," "temperature is high" and "television commercials are being aired" are good features as individual conditions, and the branching condition of "temperature is high" and "television commercials are being aired" may also be good features, but this is almost self-evident.

[0058] On the other hand, "it is winter" and "it is rich ice cream" are not very good features as individual conditions, but when combined they become good features. A combined feature that becomes good only when combined is more likely to give a user new ideas than a combination of features that are good as individual conditions. In the first embodiment, such a combined feature can be searched for as a branching condition. [Example]

[0059] In Example 2, an example will be described in which a combination feature that is highly correlated with a property is discovered in a task of estimating the property of an organic compound. Specifically, for example, Example 2 discloses an example in which a penalty is determined based on a scale different from that in Example 1, and shows a method for generating a combination feature that includes simplex conditions at a ratio desired by a user. Note that Example 2 will mainly describe the differences from Example 1, and therefore a description of the parts common to Example 1 will be omitted.

[0060] For example, consider a case where a binary variable representing the presence or absence of a functional group such as a hydroxyl group or a benzene ring is given as an explanatory variable of an organic compound and stored in explanatory variable DB221, and the melting point of the organic compound is given as an objective variable and stored in objective variable DB222.

[0061] If it is desired that the condition that a certain functional group is present appears more frequently than the condition that a certain functional group is absent among the combined features obtained as a processing result of the feature generation device 100A, a greater penalty can be imposed on the condition that a functional group is absent than on the condition that a functional group is present.

[0062] <Figure 11: Example of functional configuration of a feature generation device> 11 is a block diagram illustrating an example of a functional configuration of a feature generation device according to a second embodiment. The feature generation device 100B includes, as its functions, an interface 201, a DB 202, and a branching condition searcher 203. The interface 201 includes an input unit 211 and an output unit 212. The input unit 211 receives as input data including explanatory variables and target variables, search parameters, simplex conditions, condition groups associated with each explanatory variable, and selection ratios for each condition group.

[0063] The explanatory variables are stored in explanatory variable DB1121, the objective variables are stored in objective variable DB1122, the search parameters are stored in search parameter DB223, the individual conditions and the condition groups linked to each individual condition are stored in individual condition DB1124, and the selection ratio of each condition group is stored in condition selection ratio DB1126.

[0064] The DB 202 includes an explanatory variable DB 1121 , a response variable DB 1122 , a search parameter DB 223 , a simplex condition DB 1124 , a penalty DB 1125 , a condition selection ratio DB 1126 , and a branching condition DB 1127 .

[0065] The branching condition searcher 203 includes a conversion unit 231, a QUBO problem calculation unit 232, a branching condition generation unit 233, a penalty estimation unit 1134, and a condition determination unit 1135. The QUBO problem conversion unit 1131 converts a search problem for a branching condition that reduces the prediction error of the decision tree 400 into a QUBO problem.

[0066] <Figure 12 Explanatory variable DB1121> 12 is an explanatory diagram illustrating an example of a data structure of an explanatory variable DB 1121 according to Example 2. The explanatory variable DB 1121 is a table that stores data on explanatory variables of each sample s, and stores, as column values, a sample ID 1201 and explanatory variables associated with each sample ID 1201. Examples of the explanatory variable include a hydroxy group 1202 that is a binary variable that takes a value of 1 if a hydroxy group is present and a value of 0 if not, a benzene ring 1203 that is a binary variable that takes a value of 1 if a benzene ring is present and a value of 0 if not, and an aldehyde group 1204 that is a binary variable that takes a value of 1 if an aldehyde group is present and a value of 0 if not.

[0067] <Figure 13 Objective variable DB1122> 13 is an explanatory diagram illustrating an example of the data structure of the objective variable DB 1122 according to Example 2. The objective variable DB 1122 is a table that stores data on the objective variables of each sample s, and stores, as column values, a sample ID 1201 and an objective variable associated with each sample ID 1201. The objective variable is given as a real value, such as a melting point 1302.

[0068] <Figure 14 Single unit condition DB1124> 14 is an explanatory diagram illustrating an example of a data structure of the simplex condition DB 1124 according to Example 2. The simplex condition DB 1124 is a table that stores simplex conditions, which are elements that configure branch conditions by logical operations, and stores, as column values, IDs 1401, and simplex conditions 1402 and condition groups 1403 associated with each ID 1401.

[0069] The condition group 1403 is a group that classifies the simplex conditions 1402. For example, the value of the condition group 1403 is set to "0" for the simplex condition 1402 of an explanatory variable in which a specific substructure does not exist, and "1" for the simplex condition 1402 of an explanatory variable in which a specific substructure exists. The number of classifications of the condition group 1403 is not limited to two, and may be three or more. The condition group 1403 may be given as an integer value equal to or greater than 0 (0, 1, 2, ...), or as a character string such as "functional group present" or "functional group absent", and the data format is not limited.

[0070] <Figure 15 Penalty DB1125> 15 is an explanatory diagram illustrating an example of the data structure of the penalty DB 1125 according to the embodiment 2. The penalty DB 1125 is a table that stores penalty values ​​used when converting to QUBO problems in the second conversion unit 302, and stores, as column values, simplex conditions 1402 and penalties 1502 associated with each simplex condition 1402.

[0071] <Figure 16 Condition Selection Ratio DB1126> 16 is an explanatory diagram showing an example of the data structure of the condition selection ratio DB 1126. The condition selection ratio DB 1126 stores, as column values, condition groups 1403 and their associated selection ratios 1602. The selection ratio 1602 of a condition group 1403 is the ratio at which the single conditions 1402 included in the condition group 1403 are included in the combined feature obtained as a processing result of the feature generation device 100. This is approximately realized, for example, by the processing described below.

[0072] <Figure 17 Branching Condition DB1127> 17 is an explanatory diagram showing an example of the data structure of the branch condition DB 1127. The branch condition DB 1127 stores, as column values, a loop count 1701 and a branch condition 1702 associated therewith.

[0073] <Figure 18 Feature generation process> FIG. 18 is a flowchart of an example of a procedure for generating features according to the second embodiment.

[0074] (Step S1801) The feature generation device 100B receives input of data including explanatory variables and response variables via the input unit 211, and proceeds to step S1802. The explanatory variables are stored in the explanatory variable DB 1121, and the response variables are stored in the response variable DB 1122.

[0075] (Step S1802) The feature generation device 100B receives the search parameters 701 and the simplex conditions 1402 via the input unit 211, and proceeds to step S1803. The search parameters 701 are stored in the search parameter DB 223, and the simplex conditions 1402 are stored in the simplex condition DB 224.

[0076] (Step S1803) The feature generation device 100B receives the condition group 1403 and the selection ratio 1602 via the input unit 211, and proceeds to step S1804. The condition group 1403 is stored in the simplex condition DB 224, and the selection ratio 1602 is stored in the condition selection ratio DB 1126.

[0077] (Step S1804) The feature generation device 100B initializes the penalties 1502 stored in the penalty DB 225 to "0" for all entries, initializes the loop counter stored in the search parameter DB 223 to "0", and proceeds to step S1805.

[0078] (Step S1805) The feature generation device 100B converts the problem of searching for a branching condition that reduces the prediction error into a QUBO problem using the conversion unit 231, and proceeds to step S1806.

[0079] (Step S1806) The feature generation device 100B calculates the QUBO problem converted by the conversion unit 231 using the QUBO problem calculation unit 232, generates a branching condition 1702 from the calculation result using the branching condition generation unit 233, adds the branching condition 1702 as a new entry to the branching condition DB 1127, and proceeds to step S1807.

[0080] (Step S1807) The feature generation device 100B causes the penalty estimation unit 1134 to update the penalty 1502 in the penalty DB 225 in the following manner, and then proceeds to step S1808.

[0081] First, the feature generation device 100B adds Δwi expressed by the following equation (8) to the penalty wi of the simplex condition Ci stored in the penalty DB 225.

[0082]

number

[0083] In the above formula (8), γ is the learning rate input by the input unit 211 and stored in the search parameter DB 223. Rg is the selection ratio 1602. Furthermore, Ng is the frequency with which a single condition Ci included in the condition group g was selected in the most recent loop, and is expressed by the following formula (9).

[0084]

number

[0085] In the above formula (9), σ is the attenuation factor input by the input unit 211 and stored in the search parameter DB 223. ng(t) represents the number of simplex conditions Ci included in the condition group g selected among the branch conditions 1702 in the t loops before. The simplex conditions 1402 included in the condition group g are given by the simplex conditions 1402 and the condition group 1403 stored in the simplex condition DB 1124.

[0086] Furthermore, the simplex condition 1402 selected as the branch condition 1702 in the loop t times ago is identified from the loop counter stored in the search parameter DB 223, the loop count 1701 stored in the branch condition DB 1127, and the branch condition 1702. Specifically, for example, the loop count t times ago is calculated by subtracting t from the value of the loop counter. Therefore, by referencing the branch condition 1702 corresponding to the loop count 1701, the simplex condition 1402 selected as the branch condition 1702 in the loop t times ago is identified.

[0087] The frequency Ng is the frequency with which the single condition Ci included in the condition group g was selected in the most recent loop. For example, in the above formula (9), e -σt If there is no such condition, the frequency Ng is simply the sum of the number of times that the simple conditions Ci included in the condition group g have been selected in the loops up to now. In this embodiment, since the selection frequency of the simple conditions Ci included in the condition group g is dynamically changed while the loop is repeated, the calculation of the selection frequency Ng is performed by placing more importance on the number of times that the simple conditions Ci have been selected in the new loop than on the number of times that the simple conditions Ci have been selected in the old loop. That is, e -σt is a factor e that gets smaller in older loops. -σt By multiplying by , the frequency Ng is set to the frequency selected in the most recent loop.

[0088] (Step S1808) Feature generation device 100B uses condition determination unit 1135 to refer to the loop counter and the number of searches (100 in FIG. 7 as an example) stored in search parameter DB 223. If the current loop counter is not equal to the number of searches, it means that the search end condition is not met (step S1808: No), and the process returns to step S1806. If the current loop counter is equal to the number of searches, it means that the search end condition is met (step S1808: Yes), and the process proceeds to step S1809.

[0089] (Step S1809) The feature generation device 100B outputs the branch conditions as combination features via the output unit 212. Specifically, for example, the feature generation device 100B may output each of the branch conditions for each loop counter up to now as a combination feature, or may output the branch condition of the loop counter when the search end condition is satisfied as a combination feature.

[0090] As described above, according to Example 2, by searching for a new feature that combines a plurality of simple conditions (features) as a branching condition, it is possible to reduce the prediction error of the decision tree 400. That is, the branching condition searched for in the conventional QUBO decision tree consists only of the condition "a certain substructure does not exist" due to the bias of the simple conditions, and is not useful as a clue for considering new materials. However, according to Example 2, a branching condition that is a combination that provides new knowledge can be obtained.

[0091] In the above-described first and second embodiments, the conversion unit 231 performed the problem conversion, but the loss function H=H0+H1 of the second QUBO problem may be hard-coded in advance, and the QUBO problem calculation unit 232 may calculate the loss function H.

[0092] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

[0093] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0094] Information such as programs, tables, files, etc. that realize each function can be stored in storage devices such as memory, hard disks, SSDs (Solid State Drives), or recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0095] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be considered that almost all components are interconnected. [Explanation of symbols]

[0096] 100 Feature Generation Device 100A Feature Generation Device 100B Feature generation device 221 Explanatory Variables DB 222 Objective Variable DB 223 Search Parameter DB 224 Single Condition DB 225 Penalty DB 231 QUBO problem conversion part 232 QUBO problem calculation section 233 Branch Condition Generation Unit 301 Prediction Error Minimization Problem 302 Penalty term addition part 400 Decision Trees 701 Search parameters 802 Single Condition 902 Penalty 1131 QUBO problem conversion part 1134 Penalty Estimation Unit 1135 Condition judgment section 1403 Condition Group 1502 Penalty 1602 Selection ratio 1121 Explanatory Variable DB 1122 Objective Variable DB 1124 Single Condition DB 1125 Penalty DB 1126 Condition selection ratio DB 1127 Branching Condition DB

Claims

1. A generation device that generates branching conditions for a decision tree that estimates a target variable from a plurality of explanatory variables, the generation device comprising: a processor that executes a program; and a storage device that stores the program, the generation device comprising: The processor: an input process for inputting the plurality of explanatory variables and the response variable for each sample in a sample set, and a plurality of simplex conditions; a calculation process in which a QUBO problem is solved based on the plurality of explanatory variables and the objective variable for each sample input by the input process and the plurality of simplex conditions, the QUBO problem being a problem of calculating a first variable and a second variable that minimizes a value of a second loss function based on an error function that calculates a prediction error of the decision tree, a first loss function that calculates the sum of a first constraint and a second constraint, and a penalty for the simplex condition, the first constraint being a constraint that minimizes the first variable for each sample that indicates how many of one or more simplex conditions included in the branching condition the explanatory variable satisfies for each number of the simplex conditions satisfied by the explanatory variable, the second constraint being a constraint that minimizes the second variable for each sample that indicates whether the branching condition is one of the simplex conditions for each number of the simplex conditions satisfied by the explanatory variable, and the penalty being a term that suppresses selection to the branching condition using the second variable; a generation process of generating the branching conditions as a combination feature based on a calculation result of the calculation process; an output process for outputting the combined feature generated by the generation process; A generating device characterized by executing the above.

2. The generating device of claim 1 , In the input process, the processor inputs a plurality of condition groups into which the plurality of simple conditions are classified, and a selection ratio of the simple conditions included in each of the plurality of condition groups that are included in the combined feature, The processor: When the branching conditions are generated by the generation process, an update process is executed to update the penalty based on the selection ratio and the number of the single conditions included in the condition group that are selected among the branching conditions; The processor: execute the generation process each time the penalty is updated by the update process until a predetermined search termination condition is satisfied. A generating device characterized by:

3. A generation method executed by a generation device that has a processor that executes a program and a storage device that stores the program, and that generates branching conditions for a decision tree that estimates a target variable from a plurality of explanatory variables, comprising: The processor: an input process for inputting the plurality of explanatory variables and the response variable for each sample in a sample set, and a plurality of simplex conditions; a calculation process in which a QUBO problem is solved based on the plurality of explanatory variables and the objective variable for each sample input by the input process and the plurality of simplex conditions, the QUBO problem being a problem of calculating a first variable and a second variable that minimizes a value of a second loss function based on an error function that calculates a prediction error of the decision tree, a first loss function that calculates the sum of a first constraint and a second constraint, and a penalty for the simplex condition, the first constraint being a constraint that minimizes the first variable for each sample that indicates how many of one or more simplex conditions included in the branching condition the explanatory variable satisfies for each number of the simplex conditions satisfied by the explanatory variable, the second constraint being a constraint that minimizes the second variable for each sample that indicates whether the branching condition is one of the simplex conditions for each number of the simplex conditions satisfied by the explanatory variable, and the penalty being a term that suppresses selection to the branching condition using the second variable; a generation process of generating the branching conditions as a combination feature based on a calculation result of the calculation process; an output process for outputting the combined feature generated by the generation process; A generating method characterized by executing the following.

4. A generation program that causes a processor to generate branching conditions for a decision tree that estimates a target variable from a plurality of explanatory variables, the processor, an input process for inputting the plurality of explanatory variables and the response variable for each sample in a sample set, and a plurality of simplex conditions; a calculation process in which a QUBO problem is solved based on the plurality of explanatory variables and the objective variable for each sample input by the input process and the plurality of simplex conditions, the QUBO problem being a problem of calculating a first variable and a second variable that minimizes a value of a second loss function based on an error function that calculates a prediction error of the decision tree, a first loss function that calculates the sum of a first constraint and a second constraint, and a penalty for the simplex condition, the first constraint being a constraint that minimizes the first variable for each sample that indicates how many of one or more simplex conditions included in the branching condition the explanatory variable satisfies for each number of the simplex conditions satisfied by the explanatory variable, the second constraint being a constraint that minimizes the second variable for each sample that indicates whether the branching condition is one of the simplex conditions for each number of the simplex conditions satisfied by the explanatory variable, and the penalty being a term that suppresses selection to the branching condition using the second variable; a generation process of generating the branching conditions as a combination feature based on a calculation result of the calculation process; an output process for outputting the combined feature generated by the generation process; A generating program characterized by executing the above.

Images