Policy evaluation system, policy evaluation method, and program

Abstract

To provide a measure evaluation system capable of planning novel and further efficient measures based on execution results of tests for limited measure conditions.SOLUTION: Based on execution result information 31 obtained by executing a test on a measure object with predetermined measure conditions, a condition change tracking unit 41 determines a condition change, which is a change in measure conditions before and after the test. Based on the execution result information 31, a result verification unit 42 calculates an evaluation value obtained by evaluating the effect of the measure. A reward distribution unit 43 calculates a change contribution, which is a degree of contribution to the evaluation value of the condition change.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a policy evaluation system, a policy evaluation method, and a program.

Background Art

[0002] In enterprises and the like, in order to improve business operations, measures that act on evaluation indicators such as KPIs (Key Performance Indicators) are implemented. In recent years, not only are measures implemented, but it is also required to evaluate the effects of the implemented measures and utilize the evaluation results in formulating new measures.

[0003] Patent Documents 1 and 2 disclose a technique for conducting an AB test as a test for evaluating the effect of a measure. The AB test is a test for evaluating the effect of a measure by dividing a plurality of measure targets into two groups, implementing different measures for each group, or implementing a measure only for one group and comparing the implementation results in each group.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In order to utilize the execution results of tests such as A / B tests for formulating new measures, it is useful to conduct tests for each measure condition that the measure target has. However, in reality, due to constraints such as costs, there are limitations to the measures that can be implemented, and there may be cases where tests cannot be conducted for a sufficient number of measure conditions. In such cases, it is difficult to accurately evaluate the effect of the measure, and it is difficult to utilize the execution results of the test for formulating new measures.

[0006] The techniques described in Patent Documents 1 and 2 do not consider the fact that tests cannot be conducted for a sufficient number of measure conditions.

[0007] An object of the present invention is to provide a measure evaluation system, a measure evaluation method, and a program that enable more efficient formulation of new measures based on the execution results of tests for limited measure conditions.

Means for Solving the Problems

[0008] A measure evaluation system according to an aspect of the present disclosure includes a tracking unit that determines a condition change, which is a change in the measure conditions before and after the test, based on an execution result of a test on a measure for a measure target having a predetermined measure condition, a verification unit that calculates an evaluation value obtained by evaluating the effect of the measure based on the execution result, and a distribution unit that calculates a change contribution degree, which is a degree of contribution of the condition change to the evaluation value.

Effects of the Invention

[0009] According to the present invention, it becomes possible to formulate more efficient new measures based on the execution results of tests for limited measure conditions with respect to the evaluation value obtained by evaluating the effect of the measure.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] FIG. 1 is a block diagram showing a functional configuration of a proposal support system according to an embodiment of the present disclosure. The proposal support system 10 shown in FIG. 1 is configured by, for example, a computer system including a processor (computer) and a memory (both not shown). In this case, each component and each function of the proposal support system 10 described below are realized, for example, by the processor reading a computer program and executing the read computer program. The computer program can be recorded on a computer-readable recording medium 20. The recording medium 20 is, for example, a semiconductor memory, a magnetic disk, an optical disk, a magnetic tape, and a magneto-optical disk.

[0013] The proposal support system 10, also called a policy evaluation system, evaluates the effect of an evaluation target policy based on the execution result of a test executed on the evaluation target policy which is the policy to be evaluated, and supports the formulation of a new policy based on the evaluation result. In the present embodiment, as the evaluation target policy, an example will be described with "coupon distribution" in which a coupon (for example, a coupon for discounting the price of a target product or a coupon for making the target product free) related to a predetermined target product (for example, an onigiri) is distributed to customers who are the policy targets, but the evaluation target policy is not limited to "coupon distribution". Further, the test is an AB test in which the evaluation target policy is implemented for one of two groups obtained by dividing a plurality of policy targets, and a reference policy is implemented for the other group. The reference policy may be a no-policy in which no policy is implemented, and in the present embodiment, it is assumed that the reference policy is a no-policy.

[0014] The proposal support system 10 includes an input / output unit 1, a communication unit 2, a database 3, and a control unit 4.

[0015] The input / output unit 1 has a function of receiving various information from a user who uses the proposal support system 10 and a function of outputting various information to the user. The user is, for example, a policy implementer who has executed an AB test and a policy formulator who formulates a new policy. The policy implementer and the policy formulator may be the same.

[0016] The communication unit 2 communicates with an external system. In the example of FIG. 1, the communication unit 2 communicates with a business system 30 that performs various processes related to business via a network 40.

[0017] The database 3 stores various information used and generated by the control unit 4. In the example of FIG. 1, the database 3 stores execution result information 31, condition information 32, condition change result information 33, policy implementation result information 34, reward distribution result information 35, and policy effect prediction information 36. Hereinafter, each piece of information will be described as having a table structure, but it does not necessarily have to have a table structure.

[0018] The execution result information 31 indicates the execution result of the AB test. The condition information 32 indicates the policy conditions that are attributes of the policy target. The condition change result information 33 indicates the condition change, which is the change in the policy conditions before and after the execution of the AB test.

[0019] The policy implementation result information 34 indicates the policy evaluation result, which is the evaluation result of evaluating the effect of the policy to be evaluated. The reward distribution result information 35 indicates the calculation result obtained by calculating the degree of contribution of the condition change to the policy evaluation result for each condition change, that is, the change contribution degree. The policy effect prediction information 36 indicates the prediction result obtained by predicting the effect of a new policy candidate, which is a candidate for a new policy to be newly implemented.

[0020] Based on the execution result information 31 and the condition information 32 stored in the database 3, the control unit 4 performs an evaluation process for evaluating the effect of the policy to be evaluated and a prediction process for predicting the effect of a new policy candidate using the processing result of the evaluation process. Note that the condition change result information 33, the policy implementation result information 34, and the reward distribution result information 35 are information generated in the evaluation process, and the policy effect prediction information 36 is information generated in the prediction process.

[0021] In the example of FIG. 1, the control unit 4 includes a condition change tracking unit 41, a result verification unit 42, a reward distribution unit 43, and a new policy planning support unit 44.

[0022] The condition change tracking unit 41 is a tracking unit that determines a condition change, which is a change in the policy conditions for the policy target before and after the execution of the A / B test, based on the database 3, and generates condition change result information 33 indicating the condition change.

[0023] The result verification unit 42 is a verification unit that evaluates the effect of the policy to be evaluated based on the database 3 and generates policy implementation result information 34 indicating the policy evaluation result, which is the evaluation result.

[0024] The reward distribution unit 43 is a distribution unit that calculates, for each condition change, the degree of contribution of the condition change to the policy evaluation value result, which is the change contribution degree, based on the database 3, and generates reward distribution result information 35 indicating the calculation result.

[0025] The new policy formulation support unit 44 is a policy support unit that predicts the effect of a new policy candidate based on the database 3 and generates policy effect prediction information 36 indicating the prediction result.

[0026] Figure 2 is a diagram showing an example of the execution result information 31. The execution result information 31 shown in Figure 2 includes columns 311 to 316.

[0027] Column 311 stores the execution period during which the A / B test was executed. Column 312 stores the target ID, which is identification information for identifying the policy target. Column 313 stores the condition ID for identifying the policy conditions. Column 314 stores policy information regarding the policy implemented for the policy target by the A / B test. The policy information indicates whether the policy to be evaluated (coupon distribution) was implemented (whether the policy to be evaluated was implemented or a reference policy was implemented). In the example of Figure 2, the policy information indicates either "△ coupon", which indicates that a coupon for the target product (rice ball), which is the policy to be evaluated, was distributed, or "none", which indicates that no coupon was distributed (the policy was not implemented). Also, the set of policy targets to which coupons were distributed may be referred to as group A, and the set of policy targets to which no coupons were distributed may be referred to as group B.

[0028] Column 315 stores the KPI as an evaluation value obtained by evaluating the effect of the A / B test on the target of the measure. Column 316 stores the number of purchases of the target product (rice ball) after the A / B test (indicated as "△ number of purchases" in the figure) as the effect of the A / B test on the target of the measure. The number of purchases may be, for example, the number of purchases of the target product using a coupon, or the number of purchases of the target product within a predetermined period after the coupon is distributed. The KPI is a value calculated based on the number of purchases of the target product. The calculation formula for calculating the KPI is not limited here.

[0029] Figure 3 is a diagram showing an example of the condition information 32. The condition information 32 shown in Figure 3 includes columns 321 to 324.

[0030] Column 321 stores the condition ID for identifying the measure conditions. Column 322 stores the condition name which is the name of the measure condition. Columns 323 and 324 are provided for each variable included in the measure condition. Column 323 stores the variable name for identifying the variable, and column 324 stores the range of the variable. In the example of Figure 3, as variables, variable 1 which is the number of purchases of the target product by the target of the measure during a predetermined period before the A / B test (△ number of purchases) and variable 2 which is the age of the target of the measure are shown, but there may be other variables. Also, in the range of the variable, "(X,Y)" indicates that it is X or more and less than Y. However, when Y is "inf", it indicates that it is X or more. Note that each measure condition may have only one of variable 1 and 2. Also, the same target of the measure may have a plurality of measure conditions. For example, in the example of the figure, customers in their 30s with △ number of purchases of 5 or more and less than 10 have the measure conditions of condition IDs "C1" and "C4" respectively.

[0031] Figure 4 is a diagram showing an example of the condition change result information 33. The condition change result information 33 shown in Figure 4 is the information in which column 331 is added to the execution result information 31 (columns 311 to 316).

[0032] Column 331 stores condition change information indicating a change in the measure conditions before and after the execution of the A / B test. In the present embodiment, as the condition change information, a post-measure condition ID, which is the condition ID of the post-measure conditions that the measure target has after the execution of the A / B test, is stored. For example, when the Δ number of purchases after the execution of the A / B test for a customer having a measure condition with the condition ID "C1" is 12, the post-measure condition ID becomes "C2". Note that when the post-measure condition ID is the same as the original condition ID, it indicates that the measure conditions have not changed.

[0033] FIG. 5 is a diagram showing an example of measure execution result information 34. The measure execution result information 34 shown in FIG. 5 includes columns 341 to 346.

[0034] Column 341 stores the execution period (the same as the execution period in which the A / B test was executed) in which the measure to be evaluated was executed. Column 342 stores the condition ID of the evaluation target conditions, which are the measure conditions that the measure target has for the measure to be evaluated. Column 343 stores the measure information indicating the measure to be evaluated. Column 344 stores the number of targets, which is the number of measure targets having the evaluation target conditions for which the measure to be evaluated was executed. Columns 345 and 346 store evaluation values, which are evaluation results obtained by evaluating the effect of the measure to be evaluated. Specifically, column 345 stores the environmental contribution degree indicating the degree of contribution by the environment to the effect of the measure to be evaluated, and column 346 stores the measure contribution degree obtained by evaluating the contribution by the measure itself to the effect of the measure to be evaluated. The contribution by the environment refers to the influence on the measure to be evaluated caused by a change in the environment, apart from the measure, among the effects. The change in the environment is, for example, a change in the market situation and the natural situation.

[0035] FIG. 6 is a diagram showing an example of reward distribution result information 35. The reward distribution result information 35 shown in FIG. 6 includes columns 351 to 359.

[0036] Column 351 stores the execution period in which the measure to be evaluated was executed. Column 352 stores the condition ID of the evaluation target conditions. Column 353 stores the measure information indicating the measure to be evaluated. Column 354 stores the post-measure condition ID as the condition change information. Column 355 Store the number of policy targets with the evaluation target conditions and post - measure conditions for which the policy under evaluation was implemented (i.e., the number of policy targets for which the policy conditions changed from the evaluation target conditions to the post - measure conditions due to the policy under evaluation) as the number of targets by condition change. Column 356 stores the environmental contribution degree indicating the degree of contribution of the environment to the effect of the policy under evaluation, and column 357 stores the policy contribution degree obtained by evaluating the contribution of the policy itself to the effect of the policy under evaluation. Columns 358 and 359 store the change contribution degree, which is the degree of contribution to the policy evaluation result of the condition change. Specifically, column 358 stores the condition change environmental contribution degree, which is the degree of contribution to the environmental contribution degree of the condition change, and column 359 stores the condition change policy contribution degree, which is the degree of contribution to the policy contribution degree of the condition change.

[0037] Figure 7 is a diagram showing an example of the policy effect prediction information 36. The policy effect prediction information 36 shown in Figure 7 includes columns 361 to 370.

[0038] Column 361 stores the prediction ID for identifying the prediction result of the effect of the new policy candidate. The prediction ID includes those for identifying the prediction result of the overall effect, which is the effect of the entire new policy candidate (such as "1", "2", etc.), and the prediction result of the effect by condition change, which is the effect for each condition change among the new policy candidates (such as "1 - 1", "1 - 2", etc.). Column 362 stores the content of the new policy candidate. Column 363 stores the condition ID of the policy conditions that the implementation target of the new policy candidate has. Column 364 stores the post - measure condition ID indicating the condition change of the effect by condition change. Column 365 stores the number of targets, which is the number of policy targets for implementing the new policy candidate. Column 366 stores the overall effect, which is the effect of the entire new policy candidate. Column 367 stores the environmental contribution degree, which is the degree of contribution of the environment to the overall effect. Column 368 stores the policy contribution degree, which is the degree of contribution of the policy to the overall effect.

[0039] Column 369 stores information regarding the contribution of environmental condition changes to the overall effect. Specifically, Column 369 includes Column 369A that stores the ratio of condition changes to the whole, and Column 369B that stores the contribution of the said condition changes. Column 370 stores information regarding the contribution of condition changes due to measures to the overall effect. Specifically, Column 369 includes Column 370A that stores the ratio of condition changes to the whole, and Column 370B that stores the contribution of the said condition changes.

[0040] FIG. 8 is a sequence chart for explaining the overall processing of the proposal support system 10, and FIG. 9 is a flowchart for explaining the overall processing of the proposal support system 10.

[0041] In the overall processing, first, the input / output unit 1 executes an execution result acquisition process of receiving the execution result information 31 from the measure implementer and storing the execution result information 31 in the database 3 (step S1). Also, the input / output unit 1 executes a condition information acquisition process of receiving the condition information 32 from the measure implementer and storing the condition information 32 in the database 3 (step S2). Note that at least one of the execution result information 31 and the condition information 32 may be stored in the database 3 via the communication unit 2.

[0042] Thereafter, the condition change tracking unit 41 of the control unit 4 generates condition change result information 33 based on the execution result information 31 and the condition information 32 stored in the database 3, and executes a condition change tracking process (see FIG. 10) of storing the condition change result information 33 in the database 3 (step S3).

[0043] Subsequently, the result verification unit 42 generates measure implementation result information 34 based on the condition change result information 33 stored in the database 3, and executes a verification result process (see FIG. 11) of storing the measure implementation result information 34 in the database 3 (step S4).

[0044] Next, the reward distribution unit 43 generates reward distribution result information 35 based on the condition change result information 33 and the measure implementation result information 34 stored in the database 3, and executes a reward distribution result process (see FIG. 12) for storing the reward distribution result information 35 in the database 3 (step S5). Then, the input / output unit 1 executes a reward distribution result output process for outputting the reward distribution result information 35 stored in the database 3 (step S6).

[0045] In addition, the new measure planning support unit 44 generates measure effect prediction information 36 based on the reward distribution result information 35 stored in the database 3, and executes a new measure planning support process (see FIG. 13) for storing the measure effect prediction information 36 in the database 3 (step S7). In the new measure planning support process, the new measure planning support unit 44 can receive new measure candidates and effect prediction values from the measure planner via the input / output unit 1 or the communication unit 2, and further output the reward distribution result and the recommended values of each contribution to the measure planner.

[0046] Then, the input / output unit 1 executes a measure effect prediction output process for outputting the measure effect prediction information 36 stored in the database 3 (step S8), and ends the process.

[0047] FIG. 10 is a flowchart for explaining an example of the condition change tracking process by the condition change tracking unit 41.

[0048] In the condition change tracking process, first, the condition change tracking unit 41 acquires the execution result information 31 from the database 3 (step S101), and further acquires the condition information 32 from the database 3 (step S102).

[0049] The condition change tracking unit 41 generates condition change result information 33 by adding a column 331 for storing the post-measurement condition ID to the execution result information 31. Based on the condition information 32, for each combination of the execution period and the target ID stored in columns 311 and 312 of the execution result information 31, the condition change tracking unit 41 identifies the measure condition that the measure target identified by the target ID should meet after the execution of the A / B test as the post-measurement condition, and stores the condition ID of the post-measurement condition as the post-measurement condition ID in the added column 331 (step S103). Note that when the post-measurement condition is not defined (when it is different from the predefined measure condition), the condition change tracking unit 41 may newly define the post-measurement condition.

[0050] The condition change tracking unit 41 stores the condition change result information 33 storing the post-measurement condition ID in the database 3 (step S104), and ends the condition change tracking process.

[0051] FIG. 11 is a flowchart for explaining an example of the verification result process by the result verification unit 42.

[0052] In the verification result process, first, the result verification unit 42 acquires the condition change result information 33 from the database 3 (step S201).

[0053] The result verification unit 42 executes a loop process A that repeats the processes of steps S202 to S204 for each combination of the execution period and the condition ID in the condition change result information 33.

[0054] In the loop process A, first, based on the condition change result information 33, the result verification unit 42 sets the number of target IDs in group A for the target combination as the number of targets in group A, n A 、the number of target IDs in group B as the number of targets in group B, n B 、the average value of the KPIs in group A as the average KPI value of group A, y A 、and the average value of the KPIs in group B as the average KPI value of group B, y B and calculates them (step S202).

[0055] The result verification unit 42 calculates the average KPI value of group B, y BCalculate it as the environmental contribution degree of the measure to be evaluated (step S203).

[0056] The result verification unit 42 calculates the value obtained by subtracting the KPI average value y of group B A from the KPI average value y of group A B (y A - y B ) as the measure contribution degree of the measure to be evaluated. Then, the result verification unit 42 adds a record having the execution period and condition ID of the target combination, the measure information indicating the measure to be evaluated, and the calculated number n of targets in group A A , environmental contribution degree, and measure contribution degree to the measure implementation result information 34 (step S203).

[0057] When the result verification unit 42 finishes the processing of steps S202 to S204 for all combinations, it exits the loop processing A, stores the measure implementation result information 34 in the database 3 (step S205), and ends the verification result processing.

[0058] FIG. 12 is a flowchart for explaining an example of the reward distribution result processing by the reward distribution unit 43.

[0059] In the reward distribution process, first, the reward distribution unit 43 acquires the condition change result information 33 from the database 3 (step S301), and further acquires the measure implementation result information 34 from the database 3 (step S302).

[0060] The reward distribution unit 43 executes a loop process B that repeats the processing of steps S303 to S306 for each combination of the execution period, condition ID, and post-measure condition ID in the condition change result information 33.

[0061] In the loop process B, the reward distribution unit 43, based on the condition change result information 33, determines the number of target IDs in group A in the target combination as the number n of targets in group A A , the ratio p of the number of targets in group A to the total number of targets A , the number of target IDs in group B as the number n of targets in group B B , and the ratio p of the number of targets in group B to the total number of targets B, the average value of KPI for group A is the average value of KPI for group A y A , the average value of the KPIs for group B is the average value of the KPIs for group B B (step S303).

[0062] The reward distribution unit 43 obtains a record having the same execution period and condition ID as the execution period and condition ID of the target combination from the policy implementation result information 34, and extracts the environmental contribution ζ and policy contribution ξ included in the record (step S304).

[0063] The reward distribution unit 43 calculates the KPI average value y B minus the environmental contribution ζ (y B -ζ) is calculated as the condition change environmental contribution degree in the target combination (step S305).

[0064] The reward distribution unit 43 calculates the KPI average value y A to Group B KPI average value y B and the value after subtracting the policy contribution ξ (y A -y B The reward distribution unit 43 calculates the condition change policy contribution degree for the target combination by calculating the condition change policy contribution degree (ζ-ξ) as the condition change policy contribution degree for the target combination. The reward distribution unit 43 then calculates the condition change policy contribution degree (ζ-ξ) as the condition change policy contribution degree for the target combination. A A record having the condition change environment contribution degree and the condition change measure contribution degree is added to the reward distribution result information 35 (step S306). A is stored in the reward distribution result information 35 as the number of targets according to the change in conditions.

[0065] When the reward distribution unit 43 completes the processes of steps S303 to S306 for all combinations, it exits the loop process B, stores the reward distribution result information 35 in the database 3 (step S307), and ends the verification result process.

[0066] FIG. 13 is a flowchart for explaining an example of a new policy formulation support process by the new policy formulation support section 44.

[0067] In the new policy formulation support process, the new policy formulation support section 44 displays a new policy formulation support screen (see FIG. 16) for supporting the formulation of new policies, and receives new policy candidate information from the policy formulator via the new policy formulation support screen (step S401). The new policy candidate information includes policy information indicating a new policy candidate and policy conditions that the policy target for implementing the new policy candidate has.

[0068] Also, the new policy formulation support section 44 acquires reward distribution result information 35 from the database 3 (step S402).

[0069] The new policy formulation support section 44 acquires, from the reward distribution result information 35, each record including at least one of the variable names of the policy information and policy conditions indicated in the new policy candidate information, as the reward distribution results of similar candidates similar to the new policy candidate, and displays it on the new policy formulation support screen (step S403).

[0070] Based on the reward distribution results of the similar candidates, the new policy formulation support section 44 estimates the average value, maximum value, and minimum value, which are the statistical values of the environmental contribution degrees of the similar candidates whose variable names of the policy conditions match, as the estimated value, upper limit value, and lower limit value of the environmental contribution degree of the new policy candidate, and displays them on the new policy formulation support screen (step S404).

[0071] Based on the reward distribution results of the similar candidates, the new policy formulation support section 44 estimates the average value, maximum value, and minimum value, which are the statistical values of the policy contribution degrees of the similar candidates whose policy information matches, as the estimated value, upper limit value, and lower limit value of the policy contribution degree of the new policy candidate, and displays them on the new policy formulation support screen (step S405).

[0072] Based on the reward distribution results of similar candidates, for each condition change in similar candidates with matching policy conditions, the new policy formulation support department 44 estimates the average value, maximum value, and minimum value, which are statistical values of the condition change ratio and the environmental contribution degree of the condition change, as the estimated value, upper limit value, and lower limit value of the condition change ratio and the environmental contribution degree of the condition change in the environment of the new policy candidate, and displays them on the new policy formulation support screen (step S406). The condition change ratio is the ratio of similar candidates having the said condition change to the total number of similar candidates.

[0073] Based on the reward distribution results, for each condition change in similar candidates with matching policy information, the new policy formulation support department 44 estimates the average value, maximum value, and minimum value, which are statistical values of the condition change ratio, as the estimated value, upper limit value, and lower limit value of the condition change ratio of the new policy candidate, and displays them on the new policy formulation support screen (step S407).

[0074] Based on the reward distribution results, for each condition change in similar candidates with matching policy conditions, the new policy formulation support department 44 estimates the average value, maximum value, and minimum value, which are statistical values of the condition change policy contribution degree, as the estimated value, upper limit value, and lower limit value of the condition change policy contribution degree of the new policy candidate, and displays them on the new policy formulation support screen (step S408).

[0075] The new policy formulation support department 44 determines whether the environmental contribution prediction support button on the new policy formulation support screen has been pressed (step S409).

[0076] If the environmental contribution prediction support button has been pressed, the new policy formulation support department 44 executes the environmental contribution prediction support process (see Fig. 14) (step S410).

[0077] If the environmental contribution prediction support button has not been pressed, or if the environmental contribution prediction support process has ended, the new policy formulation support department 44 determines whether the policy contribution prediction support button on the new policy formulation support screen has been pressed (step S411).

[0078] When the policy contribution prediction support button is pressed, the new policy formulation support unit 44 executes a policy contribution prediction support process (see FIG. 14) (step S412).

[0079] When the policy contribution prediction support button is not pressed and when the policy contribution prediction support process has ended, the new policy formulation support unit 44 determines whether the condition change environment contribution prediction support button on the new policy formulation support screen has been pressed (step S413).

[0080] When the condition change environment contribution prediction support button is pressed, the new policy formulation support unit 44 executes a condition change environment contribution prediction support process (see FIG. 15) (step S414).

[0081] When the condition change environment contribution prediction support button is not pressed and when the condition change environment contribution prediction support process has ended, the new policy formulation support unit 44 determines whether the condition change policy contribution prediction support button on the new policy formulation support screen has been pressed (step S415).

[0082] When the condition change policy contribution prediction support button is pressed, the new policy formulation support unit 44 executes a condition change policy contribution prediction support process (see FIG. 15) (step S416).

[0083] When the condition change policy contribution prediction support button is not pressed and when the condition change policy contribution prediction support process has ended, the new policy formulation support unit 44 executes a prediction result correction input process (step S417). In the prediction result correction input process, the policy formulator can appropriately correct each displayed estimated value, upper limit value, and lower limit value.

[0084] After that, the New Policy Formulation Support Unit 44 determines whether the decision button has been pressed (step S418). If the decision button has not been pressed, the New Policy Formulation Support Unit 44 returns to the process of step S409. On the other hand, if the decision button has been pressed, the New Policy Formulation Support Unit 44 generates policy effect prediction information 36 based on each value calculated in steps S403 to S408 and the correction result of step S417, stores it in the database 3 (step S419), and ends the new policy formulation support process.

[0085] FIG. 14 is a flowchart for explaining an example of the environmental contribution prediction support process in step S410 of FIG. 13.

[0086] In the environmental contribution prediction support process, the New Policy Formulation Support Unit 44 acquires, as prediction target records, records that match the variables of the policy conditions of the new policy formulation candidates from the reward distribution results of similar policies (step S501). Based on the prediction target records, the New Policy Formulation Support Unit 44 calculates an aggregated value obtained by aggregating the environmental contribution degrees corresponding to the variables for each range of the variables of the policy conditions, and draws each aggregated value as a graph (step S502). The New Policy Formulation Support Unit 44 performs regression analysis on each aggregated value and adds a regression line to the graph (step S503).

[0087] The New Policy Formulation Support Unit 44 sets the average value of the environmental contribution degrees of the similar candidates as the recommended value (estimated value) of the environmental contribution degree of the new policy, calculates the range from the lower limit value to the upper limit value of the environmental contribution degrees of the similar candidates as the recommended range of the environmental contribution degree of the new policy, and displays the recommended value and the recommended range (step S504).

[0088] The New Policy Formulation Support Unit 44 determines whether a correction instruction for the recommended value and the recommended range has been received from the user (step S505). If a correction instruction has been received, the New Policy Formulation Support Unit 44 corrects the recommended value and the recommended range according to the correction instruction (step S506). After that, the New Policy Formulation Support Unit 44 determines whether the completion button has been pressed (step S507). Note that if a correction instruction has not been received, the New Policy Formulation Support Unit 44 skips step S506.

[0089] When the completion button is pressed, the New Policy Formulation Support Unit 44 ends the process if the completion button is pressed, and returns to the process of step S505 if the completion button is not pressed.

[0090] Note that the policy contribution prediction support process in step S412 of FIG. 13 is the same as the environmental contribution prediction support process described with reference to FIG. 14, and “environment” may be read as “policy”.

[0091] FIG. 15 is a flowchart for explaining an example of the condition change environment contribution prediction support process in step S414 of FIG. 13.

[0092] In the condition change environment contribution prediction support process, the New Policy Formulation Support Unit 44 obtains the average value, lower limit value, and upper limit value, which are statistical values of the condition change ratio and the condition change environment contribution degree, for each condition change in the similar candidates (step S601). For each condition change, the New Policy Formulation Support Unit 44 sets the average value of the condition change ratio and the condition change environment contribution degree as the recommended values of the condition change ratio and the condition change environment contribution degree of the new policy candidate, and draws each recommended value as a graph (step S602).

[0093] For each post-policy condition, the New Policy Formulation Support Unit 44 sets the range from the lower limit value to the upper limit value of the condition change ratio and the condition change environment contribution degree as the recommended range of the condition change environment contribution degree of the new policy candidate, and displays the recommended value and the recommended range (step S603).

[0094] The New Policy Formulation Support Unit 44 determines whether or not a correction instruction for the recommended value and the recommended range has been received from the user (step S604). If a correction instruction is received, the New Policy Formulation Support Unit 44 corrects the recommended value and the recommended range according to the correction instruction (step S605). Thereafter, the New Policy Formulation Support Unit 44 determines whether or not the completion button has been pressed (step S606). Note that if a correction instruction has not been received, the New Policy Formulation Support Unit 44 skips step S605.

[0095] When the completion button is pressed, the New Policy Formulation Support Unit 44 ends the process if the completion button is pressed, and returns to the process of step S604 if the completion button is not pressed.

[0096] Note that the conditional change policy contribution prediction support process in step S416 of FIG. 13 is the same as the conditional change environment contribution prediction support process described with reference to FIG. 15, and "environment" may be read as "policy".

[0097] FIG. 16 is a diagram showing an example of a new policy formulation support screen. The new policy formulation support screen 400 shown in FIG. 16 includes a new policy candidate selection unit 401 for inputting new policy candidate information, a distribution display unit 402 for displaying the reward distribution results of similar candidates, a policy effect display unit 403 for displaying the predicted results of the effects of new policy candidates, an environment contribution prediction support button 404, a policy contribution prediction support button 405, a conditional change environment contribution prediction support button 406, a conditional change policy contribution prediction support button 407, and a decision button 408.

[0098] FIG. 17 is a diagram showing an example of an environment contribution prediction support screen and a policy contribution prediction support screen.

[0099] The environment contribution prediction support screen 500 shown in FIG. 17(a) includes a graph 501 of the aggregated value of the environmental contribution degree of the new policy, a recommended value 502 of the environmental contribution degree of the new policy, a recommended range 503 of the environmental contribution degree of the new policy, and a completion button 504. The policy contribution prediction support screen 510 shown in FIG. 17(b) includes a graph 511 of the aggregated value of the policy contribution degree of the new policy, a recommended value 512 of the policy contribution degree of the new policy, a recommended range 513 of the policy contribution degree of the new policy, and a completion button 514.

[0100] FIG. 18 is a diagram showing an example of a conditional change environment contribution prediction support screen and a conditional change policy contribution prediction support screen.

[0101] The condition change environment contribution prediction support screen 600 shown in Fig. 18(a) includes a graph 601 of the recommended values of the condition change rate and the condition change environment contribution degree of the new policy candidate, the recommended values 602 of the condition change rate and the condition change environment contribution degree of the new policy candidate, the recommended range 603 of the condition change rate and the condition change environment contribution degree of the new policy candidate, and a completion button 604. The recommended values 602 and the recommended range 603 are displayed for each condition after the implementation of the policy.

[0102] The condition change policy contribution prediction support screen 610 shown in Fig. 18(b) includes a graph 611 of the recommended values of the condition change rate and the condition change policy contribution degree of the new policy candidate, the recommended values 612 of the condition change rate and the condition change policy contribution degree of the new policy candidate, the recommended range 613 of the condition change rate and the condition change policy contribution degree of the new policy candidate, and a completion button 614. The recommended values 612 and the recommended range 613 are displayed for each condition after the implementation of the policy. Note that the recommended values of the condition change rate and the condition change environment contribution degree of the new policy candidate may be shown together in the graph 611.

[0103] As described above, according to the present embodiment, the condition change tracking unit 41 determines a condition change, which is a change in the policy conditions before and after the test, based on the execution result information 31 of a test executed on a policy target having predetermined policy conditions. The result verification unit 42 calculates an evaluation value for evaluating the effect of the policy based on the execution result information 31. The reward distribution unit 43 calculates a change contribution degree, which is the degree of contribution to the evaluation value of the condition change. Therefore, based on the change contribution degree, it is possible to interpolate the evaluation results of the policy conditions for which the test has not been performed, and thus it is possible to formulate a more efficient new policy based on the execution results of the tests for limited policy conditions.

[0104] Also, in the present embodiment, the test is an AB test in which the policy is implemented only for group A, which is one of two groups A and B obtained by dividing a set including a plurality of policy targets. The result verification unit 42 calculates the policy implementation result information 34 showing the policy evaluation value corresponding to group A and the environment evaluation value corresponding to group B as the evaluation values. Therefore, it is possible to more appropriately evaluate the effect of the policy.

[0105] In addition, in the present embodiment, the reward distribution unit 43 calculates a policy change contribution degree, which is the degree of change contribution to the policy evaluation value, and an environmental change contribution degree, which is the degree of change contribution to the environmental evaluation value. Therefore, it becomes possible to more appropriately evaluate the degree of change contribution.

[0106] In addition, in the present embodiment, when the policy conditions after the test are different from the pre-specified policy conditions, the condition change tracking unit 41 newly specifies the policy conditions. Therefore, even if the policy conditions change to unexpected policy conditions or the like, it becomes possible to appropriately evaluate the effects of the policies.

[0107] In addition, in the present embodiment, the new policy formulation support unit 44 outputs policy effect prediction information 36 that predicts the implementation results of a new policy based on the evaluation value and the degree of change contribution. In this case, since it becomes possible to predict the effects of the new policy in advance, it becomes possible to formulate an efficient new policy.

[0108] In addition, in the present embodiment, the new policy formulation support unit 44 predicts the policy effect prediction information based on the evaluation value and the degree of change contribution of similar policies among the implemented policies, where at least one of the policy content and the policy conditions of the new policy matches. Therefore, it becomes possible to more appropriately predict the effects of the new policy.

[0109] In addition, in the present embodiment, the new policy formulation support unit 44 predicts the statistical values of the evaluation value and the degree of change contribution of the similar policies as the policy effect prediction information. Therefore, it becomes possible to more appropriately predict the effects of the new policy.

[0110] In addition, in the present embodiment, the new policy formulation support unit 44 graphs and displays the statistical values. Therefore, it becomes possible for the policy formulator who formulates the new policy to visually grasp the policy effect prediction information and appropriately evaluate the effects of the new policy.

[0111] The above-described embodiments of the present disclosure are examples for explaining the present disclosure, and are not intended to limit the scope of the present disclosure only to those embodiments. A person skilled in the art can implement the present disclosure in various other ways without departing from the scope of the present disclosure. For example, when explaining the identification information, expressions such as "identification information", "ID", "name", and "name" are used, but these can be replaced with each other. Also, although each information has been described using a table, each information may not have a table structure.

Explanation of Signs

[0112] 1: Input / output unit 2: Communication unit 2: Variable 3: Database 4: Control unit 10: Planning support system 20: Recording medium 30: Business system 40: Network 41: Condition change tracking unit 42: Result verification unit 43: Reward distribution unit 44: New policy planning support unit

Claims

1. A tracking unit that determines a condition change, which is a change in the policy conditions before and after the test, based on the execution result of a test on a policy target having predetermined policy conditions; A verification unit that calculates an evaluation value obtained by evaluating the effect of the policy based on the execution result; A distribution unit that calculates a change contribution degree, which is the degree of contribution of the condition change to the evaluation value, and has: The policy conditions are the attributes of the policy target; The execution result includes a value representing the effect for each policy target; The evaluation value is an average value of values calculated based on a predetermined calculation formula from the values representing the effects for each policy target; The test is to implement the policy only for one of two groups obtained by dividing a set including a plurality of the policy targets; The verification unit calculates a policy evaluation value, which is the evaluation value obtained by evaluating the effect of the policy corresponding to one of the two groups, and an environment evaluation value, which is the evaluation value obtained by evaluating the effect of the policy corresponding to the other of the two groups; The distribution unit calculates a policy change contribution degree, which is the change contribution degree for the policy evaluation value, and an environment change contribution degree, which is the change contribution degree for the environment evaluation value; The distribution unit calculates, for each condition change, as the environment change contribution degree, a value obtained by subtracting the environment evaluation value from the evaluation value obtained by evaluating the effect of the policy in which the condition change corresponding to the other of the two groups has occurred, and calculates, for each condition change, as the policy change contribution degree, a value obtained by subtracting, from the evaluation value obtained by evaluating the effect of the policy in which the condition change corresponding to one of the two groups has occurred, the evaluation value obtained by evaluating the effect of the policy in which the condition change corresponding to the other of the two groups has occurred and the policy evaluation value. A policy evaluation system.

2. The tracking unit newly defines the policy conditions when the policy conditions after the test are different from the predetermined policy conditions. The policy evaluation system according to claim 1.

3. The policy evaluation system further includes a policy support unit that outputs policy effect prediction information obtained by predicting the result of a new policy, which is the new policy, based on the evaluation value and the change contribution degree; The policy support unit predicts, as the policy effect prediction information, statistical values of the evaluation value and the change contribution degree of a similar policy in which at least one of the policy content and the policy conditions of the new policy coincides with those of the implemented policy, which is the policy related to the test. The policy evaluation system according to claim 1.

4. The policy support unit is the policy evaluation system according to claim 3, which graphs and displays the statistical values.

5. A policy evaluation method by a policy evaluation system, comprising: Based on the execution result of conducting a test on a policy for a policy target with predetermined policy conditions, determining a condition change that is a change in the policy conditions before and after the test; Calculating an evaluation value for evaluating the effect of the policy based on the execution result; Calculating a change contribution degree that is the degree of contribution of the condition change to the evaluation value; The policy conditions are the attributes of the policy target; The execution result includes values representing the effects for each policy target; The evaluation value is the average value of values calculated based on a predetermined calculation formula from the values representing the effects for each policy target; The test is to implement the policy only for one of two groups obtained by dividing a set including a plurality of the policy targets; In calculating the evaluation value, calculating a policy evaluation value that is the evaluation value for evaluating the effect of the policy corresponding to one of the two groups, and an environmental evaluation value that is the evaluation value for evaluating the effect of the policy corresponding to the other of the two groups; In calculating the change contribution degree, calculating a policy change contribution degree that is the change contribution degree to the policy evaluation value, and an environmental change contribution degree that is the change contribution degree to the environmental evaluation value; In calculating the policy change contribution degree, for each condition change, calculating, as the environmental change contribution degree, a value obtained by subtracting the environmental evaluation value from the evaluation value for evaluating the effect of the policy in which the condition change corresponding to the other of the two groups occurred; A policy evaluation method, wherein in calculating the environmental change contribution degree, for each condition change, calculating, as the policy change contribution degree, a value obtained by subtracting, from the evaluation value for evaluating the effect of the policy in which the condition change corresponding to one of the two groups occurred, the evaluation value for evaluating the effect of the policy in which the condition change corresponding to the other of the two groups occurred and the policy evaluation value.

6. On a computer A tracking unit that determines a condition change that is a change in the policy conditions before and after the test based on the execution result of conducting a test on a policy for a policy target with predetermined policy conditions; A verification unit that calculates an evaluation value for evaluating the effect of the policy based on the execution result; A distribution unit that calculates a change contribution degree that is the degree of contribution of the condition change to the evaluation value, wherein the policy conditions are the attributes of the policy target. The execution result includes a value representing the effect for each of the policy targets, The evaluation value is the average value of values calculated based on a predetermined calculation formula from the values representing the effects for each of the policy targets, The test is to implement the policy only for one of two groups obtained by dividing a set including a plurality of the policy targets, The verification unit calculates a policy evaluation value, which is the evaluation value obtained by evaluating the effect of the policy corresponding to one of the two groups, and an environmental evaluation value, which is the evaluation value obtained by evaluating the effect of the policy corresponding to the other of the two groups, The distribution unit calculates a policy change contribution degree, which is the change contribution degree with respect to the policy evaluation value, and an environmental change contribution degree, which is the change contribution degree with respect to the environmental evaluation value, The distribution unit calculates, for each condition change, as the environmental change contribution degree, a value obtained by subtracting the environmental evaluation value from the evaluation value obtained by evaluating the effect of the policy corresponding to the other of the two groups in which the condition change has occurred, and also calculates, for each condition change, as the policy change contribution degree, a value obtained by subtracting the evaluation value obtained by evaluating the effect of the policy corresponding to the other of the two groups in which the condition change has occurred and the policy evaluation value from the evaluation value obtained by evaluating the effect of the policy corresponding to one of the two groups in which the condition change has occurred. A program.

Images