Measure presentation device, measure presentation method, and measure presentation program

Abstract

To present which measures to take for changing a certain feature amount in a certain manner in order to improve a prediction result outputted from a prediction model.SOLUTION: A measure presenting device executes the steps of: acquiring a prediction model learnt according to a feature amount vector composed by a plurality of feature amount values, and objective variable values; setting a target objective variable value that is a target of an objective variable, and an improvement object feature amount among the plurality of feature amounts; calculating a value after change in the improvement object feature amount so as to reduce a cost which is defined on the basis of a difference in value before and after change in the improvement object feature amount and a cost measuring regulating the weight of the value before and after change, and which changes the value before and after change in the improvement object feature amount; retrieving a value after improvement in the improvement object feature amount on the basis of a prediction value of the objective variable obtained as a result of inputting the value after change in the improvement object feature amount to the prediction model and the value of the target objective variable; and outputting a retrieval result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a policy presentation device, a policy presentation method, and a policy presentation program for presenting a policy. [Background technology]

[0002] A machine learning model is a model that predicts a certain objective variable from data in which values ​​are recorded for each feature. In general, machine learning models only output the prediction of the objective variable, and do not provide any indication of how to change the prediction.

[0003] When applying machine learning models to real problems, except for problems that can be solved using only the prediction results, humans must manually consider and decide how to approach the problem based on the prediction results made by the machine. To assist with such decision-making, there has been active research into a method called Explainable AI (Artificial Intelligence), which automates not only prediction but also the analysis of the factors behind the prediction. However, Explainable AI only clarifies the factors, and does not automate how to approach the problem based on the prediction and its factors.

[0004] Therefore, there is a need to automatically derive measures to solve actual problems from a machine learning prediction model. For example, let's say there is a problem of wanting to improve the performance of a certain company. When generating a model to predict the sales of each company from past company data, what is important is not the prediction result itself of what the sales will be, but what measures can be taken to increase the sales of a company that is predicted to have a decrease or remain flat.

[0005] Such needs exist in other fields as well, such as in the medical and welfare fields, where there is a need to automatically predict not only whether a patient will recover, but also what interventions can be applied to patients that are predicted not to recover.

[0006] The following Patent Document 1 discloses a method for proposing modifications to emails so that recipients of emails take targeted actions when creating emails such as advertisements. The technology in Patent Document 1 creates a machine learning model that predicts whether or not a recipient will open an email based on the sending time and title of the email, searches for sending times and titles that will increase the probability that the recipient will open the email, and presents the sending time and title with the highest probability value.

[0007] The following Patent Document 2 discloses a data analysis device that realizes simplification of explanation of an analysis target with high accuracy and efficiency. This data analysis device uses a first neural network composed of an input layer, an output layer, and two or more intermediate layers that provide data from a previous layer and a first learning parameter to a first activation function between them to perform calculations and output the calculation results to a subsequent layer, and has a conversion unit that converts each output data to a number of dimensions of the same size based on output data from each intermediate layer and a second learning parameter, and outputs each converted output data, a rearrangement unit that rearranges the first input data in the first feature space to a second feature space based on the converted output data and the first input data in the first feature space provided to the input layer, a predicted data calculation unit that calculates predicted data for the first input data in the first feature space based on the rearrangement result and a third learning parameter, and an importance calculation unit that calculates a first importance of the first input data in each intermediate layer based on each converted output data and the third learning parameter.

[0008] Non-Patent Document 1 below discloses a method for automating the analysis of factors behind predictions made by machine learning models such as deep learning. The technology of Non-Patent Document 1 automatically analyzes factors by learning new linear regression or logistic regression based on the machine learning model that makes predictions and calculating the importance of features. Logistic regression is a machine learning model equivalent to perceptron, and is most widely used in all fields. For example, the logistic regression shown on page 119 of Non-Patent Document 2 below has a function of calculating the importance of features for the entire test sample. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] U.S. Patent No. 09317816 [Patent Document 2] JP 2019-79305 A [Non-patent literature]

[0010] [Non-Patent Document 1] Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "Why should I trust you?: Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016. [Non-Patent Document 2] Friedman J, Trevor H, Robert T. The elements of statistical learning. second edition. New York: Springer series in statistics, 2001. Summary of the Invention [Problem to be solved by the invention]

[0011] In the method of Patent Document 1, the cost of editing is extremely small due to the nature of the application, which is editing emails, so all that is required is to select the edit that will maximize the probability that the recipient will take the intended action among all the possible edits. However, in general problems such as marketing and medicine, taking any measure comes with a cost. For example, even if increasing the frequency of sales calls can increase sales, it is not possible to increase the frequency of sales calls indefinitely. Therefore, it is difficult to apply the method of Patent Document 1 directly to general problems.

[0012] The method of Non-Patent Document 1 can clarify which feature values ​​were important in the prediction results, but cannot clarify which feature values ​​should be changed and how to change them in order to change the prediction results.

[0013] The present invention aims to present measures to be taken to change which features and how to change them in order to improve the prediction results output by a prediction model. [Means for solving the problem]

[0014] a calculation process for calculating a post-change value of the improvement target feature, the post-change value being defined based on a difference between before and after a change in the improvement target feature and a cost metric specifying weights of the before and after the change, so that a cost of changing the pre-change value of the improvement target feature is small; a search process for searching for an improved value of the improvement target feature, based on a predicted value of the objective variable obtained as a result of inputting the post-change value of the improvement target feature calculated by the calculation process into the prediction model, and the value of the target objective variable; and an output process for outputting a search result obtained by the search process.

Advantages of the Invention

[0015] According to a typical embodiment of the present invention, in order to improve the prediction result output by the prediction model, it is possible to present what measures should be taken to change which feature quantities and how. Problems, configurations, and effects other than those described above will be clarified by the description of the following examples.

Brief Description of the Drawings

[0016] [Figure 1] FIG. 1 is an explanatory diagram showing a comparative example of AI. [Diagram 2] FIG. 2 is an explanatory diagram showing a configuration example of a measure presentation device that realizes an AI capable of presenting measures (C). [Diagram 3] FIG. 3 is a block diagram showing a hardware configuration example of the measure presentation device. [Figure 4] FIG. 4 is a block diagram showing a functional configuration example of the measure presentation device. [Diagram 5] FIG. 5 is a flowchart showing an example of a measure presentation processing procedure by the measure presentation device. [Figure 6] FIG. 6 is an explanatory diagram showing a first example of a setting screen by the measure presentation device. [Figure 7] FIG. 7 is an explanatory diagram showing a second example of a setting screen by the measure presentation device. [Figure 8] FIG. 8 is an explanatory diagram showing a third example of a setting screen by the measure presentation device. [Figure 9] FIG. 9 is a chart comparing the test sample with the search results in the manual mode. [Figure 10] FIG. 10 is a chart comparing the test sample with the search results in the semi-manual mode.

Modes for Carrying Out the Invention

Examples

[0017] <Comparative Example of AI> FIG. 1 is an explanatory diagram showing a comparative example of AI. In each of the graphs (A) to (C), the horizontal axis represents time, and the vertical axis represents the continuation probability, which indicates the likelihood that a customer will continue using a service. (A) A typical AI performs future predictions using a prediction model obtained through learning. That is, in the example of (A), the continuation probability is decreasing in the future prediction, and the AI ​​predicts when the customer will abandon the service due to contract cancellation.

[0018] (B) Explainable AI uses a predictive model obtained through learning to make future predictions and explains why the predictions came out that way. In the example of (B), as in (A), the probability of continuation is decreasing in the future prediction, and explainable AI is able to predict when customers will abandon the service due to contract termination, as well as explain the reasons for customer abandonment.

[0019] (C) The policy-presentable AI uses the predictive model obtained through learning to propose policies that will change the future. In the example of (C), unlike (A) and (B), the proposal of policies that will change the future increases the continuation probability in future predictions. In this way, the policy-presentable AI increases the probability of service continuation by proposing policies that will prevent customers from dropping out of the service. In the following explanation, we will explain the policy presentation device that realizes (C) the policy-presentable AI.

[0020] In the first embodiment, a policy presentation device is described as an example that uses company data up to fiscal year 2018 to create a prediction model for predicting company sales for fiscal year 2019, and presents a policy on which features of the company data should be changed and how to change them in order to increase the company's sales to a target value based on the company data and the prediction model. The policy output by the policy presentation device leads to an indication of what management policy should be adopted for the growth of the company.

[0021] <(C) Example of the configuration of a policy presentation device that realizes an AI capable of presenting policies> 2 is an explanatory diagram showing a configuration example of a policy presentation device 200 that realizes (C) a policy presentable AI. In the policy presentation device 200, a learning dataset 201 is a collection of multiple learning data, and has, as fields, an ID 210, a target variable 211, and a feature vector 212. A combination of values ​​of each field in the same row becomes an entry that constitutes one piece of learning data.

[0022] The ID 210 is identification information that uniquely identifies the learning data. For example, if the learning data is company data, the ID 210 uniquely identifies the company. N is the total number of learning data. For example, N=3000.

[0023] The objective variable 211 is ground truth data corresponding to the feature vector 212, and is a continuous value in the case of regression (the sales amount of the company in the example of the company data) and a label in the case of classification (for example, positive or negative).

[0024] The feature vector 212 is a D-dimensional vector (D is an integer equal to or greater than 1) that is a combination of the non-intervention feature vector 221 and the intervention feature vector 222. When the feature vector 212 is company data, the feature vector 212 includes, for example, company overview such as the size and business details of each company up to fiscal year 2018, capital status, management status, and the like. The feature vector 212 with ID 210 of "n" may be expressed as feature vector x(n).

[0025] The non-intervention feature vector 221 is a vector indicating the values ​​of m (m is an integer equal to or greater than 0) non-intervention features fa1 to fam (when the non-intervention features fa1 to fam are not to be distinguished from each other, they are simply referred to as non-intervention feature fa).

[0026] The non-intervention features fa1 to fam are features (e.g., the asset value of a customer) that are useful for deep learning 202 but are not intervenible in terms of improvement proposals. The intervening feature vector 222 is a vector indicating the values ​​of d (d is an integer equal to or greater than 1) intervening features fb1 to fbd (when there is no need to distinguish between the intervening features fb1 to fbd, simply referred to as the intervening feature fb).

[0027] The intervening features fb1 to fbd are features that can be intervened in the improvement proposal (for example, the loan proposal amount). When there is no need to distinguish between the intervening feature fa and the intervening feature fb, they are simply referred to as feature f.

[0028] Deep learning 202 is a process of providing a learning data set 201 to a neural network for learning and generating a prediction model 203. The prediction model 203 is a machine learning model that predicts a response variable 211 when a feature vector 212 is input. When the feature vector is corporate data, the prediction model 203 is, for example, a sales forecast model for the next fiscal year of a company based on the corporate data.

[0029] The test sample 204 is a feature vector different from the feature vector 212 included in the learning dataset 201 (it may be one of the feature vectors 212 included in the learning dataset 201). To distinguish it from the value of the ID 210 of the feature vector 212 of the learning dataset 201, the value of the ID 210 of the test sample 204 is "test" and the value of the objective variable 211 is "yt" (it may also be written as objective variable yt).

[0030] The countermeasure suggestable AI 205 uses the test sample 204 and the prediction model 205 to output countermeasure information 206 to the test sample 204. Detailed processing of the countermeasure suggestable AI 205 will be described later. The countermeasure information 206 includes, for the test sample 204, an improved objective variable 261 and an intervention possible feature vector 262 in which the objective variable yt has been improved.

[0031] The improved objective variable 261 is a predicted value output when the non-intervention feature vector 221 and the improved intervening feature vector 262 are input to the prediction model 203. The improved objective variable 261 is a prediction probability in the case of regression, and is a probability of becoming a certain label (for example, positive) in the case of classification. The improved intervening feature vector 262 includes at least one of the intervening features fb1 to fbd.

[0032] <Hardware Configuration Example of the Policy Presentation Device 200> FIG. 3 is a block diagram showing an example of a hardware configuration of the policy presentation device 200. The policy presentation device 200 includes a processor 301, a storage device 302, an input device 303, an output device 304, and a communication interface (communication IF) 305. The processor 301, the storage device 302, the input device 303, the output device 304, and the communication IF 305 are connected by a bus 306. The processor 301 controls the policy presentation device 200. The storage device 302 serves as a working area for the processor 301. The storage device 302 is a non-transient or temporary recording medium that stores various programs and data. Examples of the storage device 302 include a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), and a flash memory. The input device 303 inputs data. Examples of the input device 303 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, and a microphone. The output device 304 outputs data. The output device 304 may be, for example, a display, a printer, or a speaker. The communication IF 305 is connected to a network 310 and transmits and receives data to and from a client terminal 307.

[0033] For load balancing, the policy presentation device 200 may be configured with a plurality of devices. The client terminal 307 also has the same hardware configuration as the policy presentation device 200.

[0034] <Example of Functional Configuration of the Policy Presentation Device 200> 4 is a block diagram showing an example of a functional configuration of the policy presentation device 200. The policy presentation device 200 has, as the policy presentation capable AI 205, an acquisition unit 401, a setting input unit 402, a calculation unit 403, a search unit 404, and an output unit 405. Specifically, these are realized by, for example, causing the processor 101 to execute a program stored in the storage device 102 shown in FIG.

[0035] The acquiring unit 401 acquires the learning dataset 201 and the prediction model 203 from a storage device or an external computer. If there is no test sample in the learning dataset 201, the acquiring unit 401 acquires a test sample separately from the learning dataset 201.

[0036] The setting input unit 402 receives input of settings related to the policy of the policy. Specifically, the setting input unit 402 includes a goal input unit 421, a policy feature amount designation unit 422, and a cost information input unit 423, for example.

[0037] The goal input unit 421 inputs the value y of the objective variable that is the goal for implementing the policy. thr (hereinafter, the target variable y thr ) The target variable y thr is, for example, sales of 300 million yen. The policy feature designation unit 422 accepts designation of d (d≦D) intervening feature values ​​for implementing a policy from the feature vector 212. The designated intervening feature values ​​are referred to as "policy feature values." A policy feature value is an intervening feature value that is the target of improvement for a policy. The cost information input unit 423 accepts input of cost information required to change the policy feature values.

[0038] In this way, when data is input by the setting input unit 402, a cost metric is set for calculating the cost of changing the feature by implementing a measure. The cost metric is defined as a weight in the feature space for the value of each intervention possible feature fb before and after implementing the measure.

[0039] The calculation unit 403 calculates a changed policy feature vector based on the feature vector 212 of the learning dataset 201 acquired by the acquisition unit 401, the test sample 204, and the cost metric input by the setting input unit 402. The changed policy feature vector is a vector indicating the values ​​of one or more intervention possible feature values ​​selected from the intervention possible feature group. The changed policy feature vector is a vector that reduces the cost corresponding to the cost metric.

[0040] The search unit 404 searches for a measure that can achieve the objective variable 211 based on the changed measure feature vector calculated by the calculation unit 403. A measure that can achieve the objective variable 211 is a measure that, when the changed measure feature vector is input to the prediction model 203, the prediction result is greater than or equal to the target objective variable y thr exceeds the target variable 211, the changed policy feature vector becomes a policy that can achieve the target variable 211. The search unit 404 may search for a policy that has the lowest cost and can achieve the target variable 211, based on the changed policy feature vector calculated by the calculation unit 403.

[0041] The output unit 405 outputs the search result searched by the search unit 404 in a displayable manner. Specifically, for example, the output unit 405 displays the search result on a display of the policy presentation device 200, which is an example of the output device 304, or transmits the search result to a display of the client terminal 307 in a displayable manner.

[0042] In this way, the policy presentation device 200 calculates the changed policy feature vector based on the feature vector, the test sample 204, and the prediction model 203 read in the acquisition unit 401, and the cost metric input in the setting input unit 402, and searches for and presents the optimal policy.

[0043] <Measure presentation processing procedure> 5 is a flowchart showing an example of a procedure for a policy presentation process performed by the policy presentation device 200. The policy presentation device 200 acquires the learning dataset 201, the test sample 204, and the prediction model 203 through the acquisition unit 401 (step S501). Next, the policy presentation device 200 acquires the target objective variable y thr (Step S502). Then, the policy presentation device 200 accepts a mode selection (Step S503). The modes include a manual mode, a semi-manual mode, and an automatic mode.

[0044] The manual mode is a mode in which the user manually sets the cost metric. The semi-manual mode is a mode in which the user manually sets a part of the cost metric. The auto mode is a mode in which the policy presentation device 200 automatically sets the cost metric.

[0045] When the manual mode is selected (step S504: manual), the policy presentation device 200 accepts settings of policy features and cost metrics (step S505).

[0046] When the semi-manual mode is selected (step S504: semi-manual), the policy presentation device 200 accepts settings of policy features and information required for cost metric (cost information) (step S506).

[0047] When the auto mode is selected (step S504: auto), the policy presentation device 200 accepts the setting of the policy feature amount (step S507).

[0048] After steps S505 to S507, the policy presentation device 200 calculates a changed policy feature vector by the calculation unit 403 (step S508). Next, the policy presentation device 200 searches for a policy based on the changed policy feature vector by the search unit 404 (step S509). Finally, the policy presentation device 200 outputs the search result by the output unit 405.

[0049] <Settings screen example 1> 6 is an explanatory diagram showing a setting screen example 1 of the policy presentation device 200. In the setting screen 600, a read button 601 is a button for causing the acquisition unit 401 to read the training dataset 201, the test sample 204, and the prediction model 203 from the storage device 302.

[0050] Specifically, for example, when a read button 601 is pressed, the acquiring unit 401 reads the learning dataset 201, the test sample 204, and the prediction model 203 from the storage device 302 (step S501).

[0051] The target value input field 602 is a target objective variable y thr The value entered in the target value input field is the target objective variable y thr The target input unit 421 reads the target.

[0052] The mode selection pull-down 603 is a pull-down for selecting one of the manual mode, the semi-manual mode, and the auto mode. In Fig. 6, the case where the manual mode is selected will be described. The case where the semi-manual mode is selected will be described later in Fig. 7, and the case where the auto mode is selected will be described later in Fig. 8.

[0053] The cost metric information setting area 604 is an area for setting cost metric information when the manual mode is selected in the mode selection pull-down 603. Specifically, for example, the cost metric information setting area 604 has a policy feature selection pull-down 641, a matrix element input area 642, and a setting button 643, and receives policy feature values ​​and matrix elements of a cost metric G for calculating costs as cost metric information.

[0054] The policy feature selection pull-down 641 is a user interface for selecting an intervention possible feature fb and its number of types h (an integer satisfying h≦d). The selected intervention possible feature fb becomes the policy feature. FIG. 6 shows an example in which three intervention possible feature fb (h=3), the number of employees, the number of establishments, and the number of shareholders, are selected as policy features. The cost metric G is a matrix with h rows and h columns.

[0055] The matrix element input area 642 is an area for receiving input of element values ​​of the column of the policy feature amount. The setting button 643 is a user interface for loading the policy feature amount selected by the policy feature amount selection pull-down 641 and the element value input to the matrix element input area 642 when pressed.

[0056] The execute button 605 is a user interface that, when pressed, accepts the start of execution of processing by the calculation unit 403 and the search unit 404 using the cost metric G.

[0057] The search result display area 606 is an area for displaying the search result 660 by the search unit 404 and the improved objective variable 261. The search result 660 includes, for each policy feature, a pre-improvement value (start end side of the arrow) and a post-improvement value (end end side of the arrow). The search result 660 is the improved intervention possible feature vector 262 shown in FIG. 2.

[0058] [Cost Measurement G] Here, a detailed description will be given of the cost metric G. The cost metric G is an h×h matrix that defines a cost c shown in the following formula (1).

[0059]

number

[0060] Vector x org is an h-dimensional vector that indicates the policy features before the change, and vector x actis an h-dimensional vector that indicates the policy feature quantity after the change. det(G) is the determinant of the cost metric G. The above formula (1) expresses the difference vector of the policy feature quantity (vector x act -vector x org ) and define the cost c by the sum of their products. When taking this sum, that is, when measuring the cost c, act -vector x org The cost metric G specifies the weight of each product of differences in

[0061] The calculation unit 403 calculates the vector x that minimizes the cost c by using the formula (1). act The calculation unit 403 calculates the vector x act A set of vectors x with cost c less than or equal to a threshold may be calculated. act A group may be calculated.

[0062] [Search process] Next, the search process by the search unit 404 will be described in detail. The search unit 404 performs a measure to minimize the loss function L in the following formula (2), specifically, the vector x act Explore.

[0063]

number

[0064] In the above formula (2), σ is a sigmoid function, a is a parameter that determines the slope of the sigmoid function σ, and λ is a parameter that determines the weight of the cost c, and each can be set arbitrarily. In this example, a=1000, λ=0.1. The improved objective variable y act is the vector x org to vector x actThe non-intervention feature vector 221 replaced by and the feature vector combined with are input to the prediction model 203. Note that in the above formula (2), the sigmoid function σ may be replaced with a linear function, a ReLU function, or a GeLU function.

[0065] The first term on the right-hand side of the loss function L is the objective variable y act is the target variable y thr If it exceeds L, L=-1. If it does not exceed L, L=0. act is the target variable y thr This can be interpreted as giving a loss of L=1 if the loss does not exceed .

[0066] The second term on the right-hand side of the loss function L is a term proportional to the cost c. By minimizing the sum of the first and second terms on the right-hand side, the objective variable y act is the target variable y thr A vector x that indicates a measure that exceeds and has a small cost c act The second term on the right hand side can be omitted.

[0067] The search unit 404 may search for a measure with the second smallest or smaller loss function L, instead of the measure with the smallest loss function L. The user can select which measure to adopt. For example, the user can select a measure according to the smallness of the loss function L using the measure selection pull-down 661. In this way, the search unit 404 searches for the measure selected in the measure selection pull-down 661. In the example of FIG. 6, "2nd best action" is selected using the measure selection pull-down 661.

[0068] The output unit 405 outputs the finally obtained vector x act The values ​​of (890, 36, 390) and the vector x orgThe values ​​(780, 25, 420) of the loss function L are output so as to be displayable in the search result display area 606 as the search result 660. In the example of FIG. 6, since “2nd best action” is selected in the policy selection pull-down 661, the policy with the second smallest value of the loss function L is searched for and displayed as the search result 660.

[0069] <Settings screen example 2> Fig. 7 is an explanatory diagram showing a setting screen example 2 of the policy presentation device 200. Fig. 7 shows an example of the setting screen when the semi-manual mode is selected in the mode selection pull-down 603. When the semi-manual mode is selected, a policy feature selection pull-down 641, a cost information input area 701, a policy feature pair input area 702, and a setting button 643 are displayed in the cost metric information setting area 604.

[0070] In the setting screen 700, a cost information input area 701 is a user interface that accepts input of cost information. Cost information is weights required when moving a policy feature independently. Specifically, for example, if the weight for increasing the "number of employees" by one is 1 and the weight for increasing the number of establishments by one is 10 (unit: 10 million yen), then "1" and "10" are input as cost information in the cost information input area 701, respectively.

[0071] Moreover, the cost information is set in the diagonal elements of the cost metric G. That is, in the manual mode shown in Fig. 6, it was possible to input all matrix elements of the cost metric G, but in the semi-manual mode, it is possible to input only the diagonal elements of the cost metric G.

[0072] The policy feature pair input area 702 is an area for receiving input of a policy feature pair. Specifically, for example, the policy feature pair input area 702 has a same-direction policy feature pair input area 721 and an opposite-direction policy feature pair input area 722.

[0073] The same-direction policy feature pair input area 721 is an area for accepting input of a same-direction policy feature pair. A same-direction policy feature pair is a policy feature pair that shows a negative correlation, that is, a policy feature pair that the user judges to be difficult to increase when one of the two policy feature pairs is increased. For example, if it is difficult to increase "number of employees" when "establishments" are increased, the user may input (number of employees, number of establishments) as a same-direction policy feature pair in the same-direction policy feature pair input area 721.

[0074] The opposite direction policy feature pair input area 722 is an area that accepts input of a policy feature pair that shows a positive correlation, that is, an opposite direction policy feature pair. An opposite direction policy feature pair is a policy feature pair that the user judges to be difficult to decrease when one of the two policy feature pairs is increased. For example, if it is difficult to decrease "number of employees" when "establishments" is increased, the user can input (number of employees, number of establishments) as an opposite direction policy feature pair in the opposite direction policy feature pair input area 722.

[0075] If there are no particular restrictions as described above, the user does not need to input anything into the policy feature pair input area 702 .

[0076] As shown in Figure 7, if "number of employees" and "number of establishments" are selected as policy features, "1" and "10" are entered respectively in the cost information input area 701 for each policy feature, (number of employees, number of establishments) are entered in the opposite direction policy feature pair input area 722, and nothing is entered in the same direction policy feature pair input area 721, the cost metric G is calculated as follows:

[0077] First, cost measurement G ort is expressed by the following equation (3).

[0078]

number

[0079] Cost Measurement G ort The cost information input in the cost information input area 701 is set in the diagonal elements of and 0 is set in the off-diagonal elements.

[0080] The calculation unit 403 calculates the cost metric G ort When the cost c in the above formula (1) is calculated using the above, the cost c is the sum of the square of the difference between the values ​​of the policy feature "number of employees" before and after the policy and the square of the difference between the values ​​of the policy feature "number of establishments" before and after the policy multiplied by 10. This means that changing the policy feature "number of establishments" imposes a cost c that is 10 times that of changing the policy feature "number of employees." In this case, the cost c is proportional to the sum of the squares of the major axis and the minor axis of an ellipse with major axis:minor axis = √10:1 on a two-dimensional plane with the difference between the values ​​of "number of employees" and "number of establishments" before and after the policy on the horizontal and vertical axes, respectively.

[0081] Then, the calculation unit 403 calculates the cost metric G ort is rotated using a rotation matrix to calculate the cost metric G.

[0082]

number

[0083] The above formula (4) corresponds to stretching an ellipse with major axis:minor axis = √10:1 by a factor of α along the horizontal axis and then rotating it by -θ degrees. Here, the parameters α = 1.5 and θ = 45°. This transforms the ellipse into a shape stretched in the -45 degree and 135 degree directions. This means that the cost c, that is, the sum of the squares of the major axis and minor axis of this ellipse, will increase when the difference between the values ​​of "number of employees" and "number of establishments" before and after the measure is positive and negative, or negative and positive, respectively.

[0084] On the other hand, if you enter (number of employees, number of establishments) in the same-direction policy feature pair input area 721 and do not enter anything in the opposite-direction policy feature pair input area 722, that is, if you want to impose a penalty on policies that increase the "number of establishments" when the "number of employees" increases, or decrease the "number of establishments" when the "number of employees" decreases, simply replace θ in the above formula (4) with -θ.

[0085] In this case, the ellipse is stretched in the 45 degree and 225 degree directions, and the cost c becomes larger when the difference between the values ​​of "number of employees" and "number of establishments" before and after the policy is positive and positive, or negative and negative (i.e., when they have the same sign).

[0086] When considering policy features and cost metric G in three or more dimensions, just input cost information for the diagonal components in the cost information input area 701 as in the two-dimensional case, and perform the same operation as in formula (4) above for each policy feature pair input in the policy feature pair input area 702.

[0087] Using the cost metric G thus obtained, the calculation unit 403 calculates the vector x that minimizes the cost c by the above formula (1), in the same manner as in the case described with reference to the setting screen example 1 of FIG. act The search unit 404 calculates the vector x act We will use this to search for a measure to minimize the loss function L in the above equation (2).

[0088] <Settings screen example 3> Fig. 8 is an explanatory diagram showing a setting screen example 3 of the policy presentation device 200. Fig. 8 is an example of the setting screen when the auto mode is selected in the mode selection pull-down 603. When the auto mode is selected, a policy feature selection pull-down 641 and a setting button 643 are displayed in the cost metric information setting area 604 of the setting screen 800. Furthermore, in the auto mode, all intervention possible features fb become policy features unless otherwise specified by the user.

[0089] Since the auto mode is selected, the policy presentation device 200 automatically determines the information set in step 506 in the semi-manual mode by the policy feature designation unit 422 and the cost information input unit 423 based on the correlation between the intervention possible features in the learning dataset 201. Specifically, for example, in the auto mode, the policy feature designation unit 422 calculates the correlation coefficient between two policy feature values ​​in a brute force manner, and sets an intervention possible feature pair showing a positive correlation as an opposite direction policy feature pair, and sets an intervention possible feature pair showing a negative correlation as a same direction policy feature pair.

[0090] For example, suppose that there are three possible intervention features fb: "number of employees," "number of establishments," and "number of shareholders." When the variances of these possible intervention features fb are "72," "4.7," and "18," respectively, the correlation between "number of employees" and "number of establishments" is positively large, and the correlations between the other possible intervention feature pairs are all negative, the cost information input unit 423 sets the cost weight G to three significant digits as shown in the following formula (5).

[0091]

number

[0092] Note that a positive threshold value equal to or greater than 0 and smaller than 1, and a negative threshold value equal to or greater than -1 and smaller than 0 may be set. In this case, the policy presentation device 200 may set an intervention possible feature pair whose correlation coefficient is equal to or greater than the positive threshold value as an opposite-direction policy feature pair, and may set an intervention possible feature pair whose correlation coefficient is equal to or less than the negative threshold value as a same-direction policy feature pair.

[0093] Then, the calculation unit 403 calculates the vector x act The following calculation is made:

[0094] Thus, according to the first embodiment, by inputting information on policy features and their cost metrics in the setting screens 600, 700, and 800, the policy presentation device 200 can present policies on which policy features of the data should be changed and how, in order to increase the company's sales to a target value. EXAMPLES

[0095] In the second embodiment, an example will be described in which the policy presentation device 200 of the first embodiment creates a machine learning model for predicting house prices from Boston house price data, and presents what kind of policy is necessary to raise house prices to a target value. The Boston house price data is a data set consisting of regional information such as the crime rate of the area where the house is located and ease of access to the highway, and data on the house itself such as the average number of rooms in the house and the amount of property tax. The policy output by the policy presentation device 200 based on this embodiment makes it possible to clarify how to increase the real estate value of a house.

[0096] The following describes the results of checking the operation of the policy presentation device 200 using Boston home price data. The Boston home price data is a learning dataset 201 with N=506 samples, D=13-dimensional feature set, and home price (in thousand dollars) as the objective variable.

[0097] As the prediction model 203 for predicting housing prices, the prediction model 203 described in Patent Document 2 was used. The accuracy was a coefficient of determination R2=0.830. Note that, instead of the prediction model 203 described in Patent Document 2, any other machine learning model may be used. For example, a linear regression model, a random forest model, a multilayer perceptron model, or the like may be used.

[0098] Of the intervention-enabled features of the Boston home price data, the average number of rooms RM, the highway access index RAD, and the property tax TAX are adopted as policy features. The highway access index RAD is a natural number variable between 1 and 24, indicating that the higher the index, the better the access. The test sample 204 for which a policy is proposed is a house with a home price of "27.5", an average number of rooms RM = 6.727, a highway access index RAD = 5, and a property tax TAX = 384. The aim is to achieve a home price of 32.5, which is the target objective variable, by changing the policy features of this test sample 204.

[0099] Below, examples of data input in each mode and the search results are shown along the procedures for the manual mode and semi-manual mode in Example 1. Here, the values ​​of the parameters used in the above formulas (1) to (5) are the same as those in Example 1, and values ​​normalized to a mean of 0 and a variance of 1 are used as data corresponding to the policy feature quantities in the above formulas (1) to (5) instead of the original values ​​of the policy feature quantities.

[0100] <Manual mode> Consider the case where cost metrics G1 and G2 are input as shown in formula (6) below.

[0101]

number

[0102] Cost metric G1 assumes that the cost information associated with changes in all policy features is the same (all diagonal elements are 1), while cost metric G2 assumes that the cost information for the highway access index RAD and the fixed asset tax TAX are twice and half, respectively, compared to the cost information associated with changes in the average number of rooms RM.

[0103] Using cost metric G1, the best policy feature values ​​(smallest loss function L) were (RM=7.594, RAD=6, TAX=323.2), and the predicted value of the objective variable, housing price, was 32.60.

[0104] Using the cost metric G2, the best policy feature values ​​were (RM=7.609, RAD=5, TAX=311.9), and the predicted value of the objective variable, housing price, was 32.80.

[0105] Figure 9 is a chart comparing the search results of the test sample 204 and the manual mode. Comparing the search results when using cost metrics G1 and G2, the following can be seen. First, in both cases, the house price exceeds the target objective variable value of 32.5, and the values ​​of the optimal policy features (RM, RAD, TAX) are not significantly different from the policy feature values ​​(6.727, 5,384) of the test sample 204, and it can be seen that the goal is being achieved while minimizing the cost c.

[0106] In addition, when comparing the values ​​of the optimal policy features (RM, RAD, TAX) for cost metrics G1 and G2, the average number of rooms RM is almost the same, while the highway access index RAD increases by 1 from the policy feature value of “5” for test sample 204 in cost metric G1, but takes the same value as the policy feature value of “5” for test sample 204 in cost metric G2.

[0107] Furthermore, the difference between the property tax value "311.9" in cost metric G2 and the policy feature value "384" in the test sample 204 is larger than the difference between the property tax value "323.2" in cost metric G1 and the policy feature value "384" in the test sample 204. In other words, it can be seen that the cost c is calculated by more correctly reflecting the assumptions for cost metric G2 than for cost metric G1.

[0108] <Semi-manual mode> Assume that in the cost information input area 701, the average number of rooms RM, the access index RAD to the highway, and the fixed property tax TAX are all set to "1," (RM, RAD) is input in the same-direction policy feature pair input area 721, and (RM, TAX) is input in the opposite-direction policy feature pair input area 722. In this case, the best policy feature values ​​are RM=7.73, RAD=3, TAX=410, and the predicted value of the housing price, which is the objective variable, is 32.7.

[0109] 10 is a chart comparing the test sample 204 with the search results in the semi-manual mode. That is, compared to the test sample 204, RM is larger, RAD is smaller, and TAX is larger. This is a result that correctly reflects the input of the same-direction policy feature pair input area 721 and the opposite-direction policy feature pair input area 722, which imposes cost information for both the average number of rooms RM and the highway access index RAD becoming larger or smaller, and for one of the average number of rooms RM and the fixed property tax TAX becoming larger and the other becoming smaller.

[0110] If the same-direction policy feature pair input area 721 and the opposite-direction policy feature pair input area 722 are left blank, the search results will be the same as those of the cost metric G1 in manual mode. However, by entering data into the same-direction policy feature pair input area 721 and the opposite-direction policy feature pair input area 722, it can be seen that a result completely different from the search result of the cost metric G1 is obtained. EXAMPLES

[0111] In the third embodiment, an example will be described in which the policy presentation device 200 of the first embodiment creates a prediction model 203 for predicting whether or not a loan can be granted from data of a loan applicant, and presents what policy should be implemented to be determined as loanable. The results output by the policy presentation device 200 based on this embodiment can show a customer who has been determined as not eligible for a loan how to change his / her income, application details, etc. to make the loan eligible, thereby enabling more people to use loans.

[0112] The test sample 204 is time series data defined by a combination of a feature vector xn indicating each value of the feature group and a response variable yn indicating a future value to be predicted. The value n=1,...,N of the ID 210 is an index for specifying a loan applicant.

[0113] The feature vector xn is a D-dimensional real-valued vector, and includes application information such as the loan applicant's past credit information, income information, and the face value and details of the loan applied for. The goal variable yn takes a class value of either 0 or 1. In this embodiment, the goal variable yn=1 means that the loan is determined to be unlendable, and the goal variable yn=0 means that the loan is determined to be lendable.

[0114] By executing a policy presentation procedure similar to that of the first embodiment, the policy presentation device 200 can present to a customer who is not eligible for a loan what and how much of the credit information, income information, and application information should be changed to reduce costs and to make the customer eligible for a loan.

[0115] Unlike the regression model, the classification model in the third embodiment predicts the probability value of being classified into each class. Therefore, the improved objective variable y act corresponds to the probability value of being classified into the target class, so the target objective variable y thr is the threshold value of the probability value for classifying into the target class. Here, instead of the probability value itself, the reciprocal of the amount of information contained in the probability value p, "-log(p)", or "log(p)" with a negative sign may be used.

[0116] In this way, the policy presentation device 200 can present which policy should be taken to change which feature quantity and how to change it in order to change the prediction result at the lowest cost.

[0117] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Also, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Also, a configuration of another embodiment may be added to a configuration of one embodiment. Also, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0118] Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0119] Information such as programs, tables, files, etc. that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0120] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0121] 101 Processor 102 Storage Devices 200 Measure presentation device 201 Training Dataset 202 Deep Learning 203 Prediction Model 204 Test Sample 205 Predictive Model 211 Objective Variable 212 Feature Vector 221 Non-intervention feature vector 222 Intervention-enabled feature vector 301 Processor 302 Storage Devices 401 Acquisition Department 402 Setting input section 403 Calculation Department 404 Search Department 405 Output section 421 Target Input Section 422 Policy feature quantity specification part 423 Cost information input section

Claims

1. A policy presentation device having a processor that executes a program and a storage device that stores the program, The processor, An acquisition process of acquiring a prediction model trained using a feature vector composed of a plurality of feature values ​​and a target variable value; a setting process for setting a value of a target objective variable that is a target of the objective variable and an improvement target feature value among the plurality of feature values; a calculation process for calculating a post-change value of the improvement target feature, the post-change value being defined based on a difference between values ​​of the improvement target feature before and after the change and a cost metric specifying weights of the values ​​before and after the change, such that a cost of changing the pre-change value of the improvement target feature is reduced; a search process for searching for an improved value of the improvement-target feature quantity based on a predicted value of the objective variable obtained as a result of inputting the changed value of the improvement-target feature quantity calculated by the calculation process into the prediction model and the value of the target objective variable; an output process for outputting a search result obtained by the search process; A policy presentation device that executes the above.

2. The policy presentation device according to claim 1, In the setting process, the processor sets the improvement target feature by accepting a selection of the improvement target feature from among the plurality of feature amounts. A policy presentation device comprising:

3. The policy presentation device according to claim 1, In the setting process, the processor sets the improvement target feature by receiving an input of the cost metric. A policy presentation device comprising:

4. The policy presentation device according to claim 1, In the setting process, the processor sets the cost metric by receiving an input of a part of weights that increase or decrease the value of the improvement target feature. A policy presentation device comprising:

5. The policy presentation device according to claim 4, In the setting process, the processor receives an input of a pair of improvement target features exhibiting a negative correlation, and sets the cost metric using the partial weight and a remaining weight that increases the cost when a difference between a value before and after the change for one of the pair of improvement target features and a value before and after the change for the other of the pair of improvement target features have values ​​with different signs. A policy presentation device comprising:

6. The policy presentation device according to claim 4, In the setting process, the processor receives an input of a pair of improvement target features showing a positive correlation, and sets the cost metric using the partial weight and a remaining weight that increases the cost when a difference between a value before and after the change for one of the pair of improvement target features and a difference between a value before and after the change for the other of the pair of improvement target features have values ​​of the same sign. A policy presentation device comprising:

7. The policy presentation device according to claim 1, In the setting process, the processor sets the cost metric based on a correlation between the improvement target features. A policy presentation device comprising:

8. The policy presentation device according to claim 1, In the calculation process, the processor calculates a changed value of the improvement target feature that minimizes the cost. A policy presentation device comprising:

9. The policy presentation device according to claim 1, In the search process, the processor searches for an improved value of the improvement target feature based on the predicted value of the objective variable, the value of the target objective variable, and the cost. A policy presentation device comprising:

10. The policy presentation device according to claim 1, In the search process, when the predicted value of the objective variable exceeds the value of the target objective variable, the processor determines a post-change value of the improvement-target feature quantity as a post-improvement value of the improvement-target feature quantity. A policy presentation device comprising:

11. The policy presentation device according to claim 1, In the output process, the processor outputs, as the search result, a value of the improvement target feature quantity before the change and a value of the improvement target feature quantity after the change in the search result in a displayable manner. A policy presentation device comprising:

12. The policy presentation device according to claim 1, In the output process, the processor outputs, in a displayable manner, a predicted value that is output as a result of inputting the improved value of the improvement target feature into the prediction model. A policy presentation device comprising:

13. A method for presenting a policy using a policy presentation device having a processor that executes a program and a storage device that stores the program, comprising: The processor, An acquisition process of acquiring a prediction model trained using a feature vector composed of a plurality of feature values ​​and a target variable value; a setting process for setting a value of a target objective variable that is a target of the objective variable and an improvement target feature value among the plurality of feature values; a calculation process for calculating a post-change value of the improvement target feature, the post-change value being defined based on a difference between values ​​of the improvement target feature before and after the change and a cost metric specifying weights of the values ​​before and after the change, such that a cost of changing the pre-change value of the improvement target feature is reduced; a search process for searching for an improved value of the improvement-target feature quantity based on a predicted value of the objective variable obtained as a result of inputting the changed value of the improvement-target feature quantity calculated by the calculation process into the prediction model and the value of the target objective variable; an output process for outputting a search result obtained by the search process; A method for presenting a measure, comprising:

14. The processor: An acquisition process of acquiring a prediction model trained using a feature vector composed of a plurality of feature values ​​and a target variable value; a setting process for setting a value of a target objective variable that is a target of the objective variable and an improvement target feature value among the plurality of feature values; a calculation process for calculating a post-change value of the improvement target feature, the post-change value being defined based on a difference between values ​​of the improvement target feature before and after the change and a cost metric specifying weights of the values ​​before and after the change, such that a cost of changing the pre-change value of the improvement target feature is reduced; a search process for searching for an improved value of the improvement-target feature quantity based on a predicted value of the objective variable obtained as a result of inputting the changed value of the improvement-target feature quantity calculated by the calculation process into the prediction model and the value of the target objective variable; an output process for outputting a search result obtained by the search process; A policy presentation program that causes a program to execute the above steps.

Images