Data analysis support device and data analysis support method

Abstract

To enable a selection of an optimal threshold in learning model for binary classification.SOLUTION: A data analysis support apparatus 100 includes an accuracy optimization unit 112 that accesses prediction value data (prediction value database 130) in which a prediction value indicating a probability of an event pertaining to a prediction target and a correct answer indicating correctness or incorrectness of an event pertaining to the prediction target. The accuracy optimization unit 112 receives a lower limit setting accuracy, which is a prediction accuracy for which the lower limit is set of a fit rate and a reproducibility rate for predicting that an event will occur if the prediction value of the prediction target is a threshold value or more, and calculates an accuracy optimization threshold value which is a threshold value at which the lower limit setting accuracy is the lower limit or more and an optimized target accuracy which is the prediction accuracy different from the lower limit setting accuracy among the fit rate and the reproducibility rate is maximized.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a data analysis support device and a data analysis support method for optimizing parameters related to machine learning that predicts events based on data. [Background technology]

[0002] With the advancement of computer-based processing of large amounts of data and machine learning technology, it is now possible to predict events based on data. Examples of predictions using machine learning technology include predicting failures from measurement data on equipment and machinery, predicting service contract cancellations from service usage status, and predicting (diagnosing) illnesses from test data.

[0003] To utilize machine learning technology, a machine learning model (learning model, prediction model) is required. In the inspection device described in Patent Document 1, a learning model is created every time a certain number of cycles of calculations are performed in a learning cycle called an epoch or learning iteration of a neural network having three or more layers, and an ROC (Receiver Operating Characteristic) curve is used as a method for evaluating the accuracy of the multiple learning models created, thereby enabling the selection of the optimal learning model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-042669 Summary of the Invention [Problem to be solved by the invention]

[0005] If highly accurate predictions can be made using a learning model for binary classification, such as failure / normal and cancellation / continuation, it will be possible to take measures (responses) before untoward events such as failures or cancellations occur. For example, it will be possible to explain the benefits to customers who are likely to cancel their contracts and encourage them to continue, or to replace parts that are predicted to fail during the next regular inspection / downtime. This will allow services to continue, ensuring profits, and preventing machine downtime due to failures.

[0006] To make highly accurate predictions, it is important to adjust the parameters of the learning model. In particular, it is important to set a threshold value for determining whether a failure or cancellation has occurred for predicted values such as breakdowns and cancellations. The technology described in Patent Document 1 makes it possible to select the optimal learning model from multiple created learning models, but the optimal threshold value for a single learning model remains unknown. The present invention has been made in view of the above background, and an object of the present invention is to provide a data analysis support device and a data analysis support method that enable selection of an optimal threshold in a learning model for binary classification. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the data analysis support device of the present invention includes an accuracy optimization unit that accesses prediction value data in which a prediction value indicating the probability that an event related to the prediction target will occur is associated with a correct answer indicating whether the event related to the prediction target is correct, receives a lower limit setting accuracy, which is a prediction accuracy at which a lower limit is set among the precision and recall when predicting that the event will occur if the prediction value of the prediction target is equal to or greater than a threshold, and calculates an accuracy optimization threshold, which is the threshold at which the lower limit setting accuracy is equal to or greater than the lower limit and at which the optimization target accuracy, which is a prediction accuracy different from the lower limit setting accuracy among the precision and recall, is maximized. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a data analysis support device and a data analysis support method that enable selection of an optimal threshold in a learning model for binary classification. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining a churn prediction device according to an embodiment of the present invention. [Figure 2] 1 is a diagram for explaining an overview of a data analysis support device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a functional block diagram of the data analysis support device according to the present embodiment. [Figure 4] FIG. 2 is a data configuration diagram of a predicted value database according to the embodiment. [Figure 5] FIG. 10 is a data configuration diagram of a database of the number of subscribers by predicted value according to the present embodiment. [Figure 6] 10 is a flowchart of an accuracy optimum threshold calculation process according to the present embodiment. [Figure 7] FIG. 10 is a diagram illustrating the screen configuration of an accuracy optimum threshold screen according to the present embodiment. [Figure 8] 10 is a flowchart of an effect optimum threshold calculation process according to the present embodiment. [Figure 9] FIG. 10 is a screen configuration diagram of an effect calculation parameter screen according to the present embodiment. [Figure 10] FIG. 10 is a diagram illustrating the screen configuration of an improvement effect screen according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] <<Outline of the data analysis support device>> A data analysis support device in a mode (embodiment) for carrying out the present invention will be described below. FIG. 1 is a diagram for explaining a churn prediction device 500 according to this embodiment. The churn prediction device 500 uses a prediction model 521, which is a machine learning model used for prediction, to calculate the probability (predicted probability, predicted value) that a subscriber will churn (the event of churn occurring / happening) based on the subscriber's service usage status, such as the start time of service usage and frequency of use. The churn prediction device 500 predicts churn (occurrence of an event) if the predicted value is equal to or greater than a predetermined threshold 522, and predicts continuation if the predicted value is less than the threshold.

[0011] 2 is a diagram illustrating an overview of the data analysis support device 100 according to this embodiment. A prediction model 521 (see FIG. 1) of the churn prediction device 500 is generated using training data 560. The training data 560 includes identification information of a subscriber, a service usage status, and a correct answer (correct answer label) indicating whether the subscriber has continued or canceled the service. The usage status is an explanatory variable (input) of the prediction model 521, and the correct answer is a target variable (output). The input data of data analysis support device 100 is predicted value data 570, which includes predicted values and correct answers for cancellation. Data analysis support device 100 accepts the lower limit of either the precision or the recall rate with respect to the accuracy of prediction, and calculates a threshold value of the predicted value (optimal accuracy threshold) that maximizes the other rate.

[0012] According to the data analysis support device 100, a threshold value for the predicted value that maximizes the precision or recall of the prediction model 521 required to achieve the user's business performance goal can be obtained while satisfying the lower limit of either the precision or recall. Even a user without knowledge of machine learning or data science can set an optimal threshold value for the prediction model 521 according to the purpose of use.

[0013] The user of the data analysis support device 100 conducts sales activities to encourage subscribers who are predicted to cancel their contracts to continue. By conducting sales activities, some subscribers may be able to continue the service and make a profit, but the sales activities themselves incur costs. There is a trade-off between the profits from sales activities and the costs (losses) involved. The data analysis support device 100 receives the profit (expected profit value) and cost of sales activities and calculates a threshold value that maximizes the effect of subtracting the cost from the profit. The user can obtain a prediction probability threshold value (effectiveness-optimizing threshold value) that maximizes the effect / efficiency of the business, not just from the viewpoint of accuracy.

[0014] <Data analysis support device: Overall configuration> 3 is a functional block diagram of a data analysis support device 100 according to this embodiment. The data analysis support device 100 is a computer, and includes a control unit 110, a storage unit 120, and an input / output unit 180. User interface devices such as a display, keyboard, and mouse are connected to the input / output unit 180. The input / output unit 180 may include a communication device, enabling data transmission and reception with devices such as the churn prediction device 500. A media drive may also be connected to the input / output unit 180, enabling data exchange using a recording medium.

[0015] The storage unit 120 is configured to include storage devices such as a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), etc. The storage unit 120 stores a forecast value database 130, a forecast value-specific subscriber number database 140, and a program 128. The program 128 includes descriptions of an accuracy optimum threshold calculation process (see FIG. 6) and an effectiveness optimum threshold calculation process (see FIG. 8), which will be described later.

[0016] <Data analysis support device: predicted value database> 4 is a data configuration diagram of the predicted value database 130 according to this embodiment. Data stored in the predicted value database 130 is input data for the accuracy optimum threshold calculation process (see FIG. 6) described below. The predicted value database 130 is data in a tabular format, and one row (record) includes columns of identification information 131, predicted value 132, and correct answer 133. In other words, in the predicted value database 130, the predicted value 132 and the correct answer 133 are associated with each other.

[0017] The identification information 131 is identification information of the subscriber. The predicted value 132 is a predicted value (predicted probability) that the subscriber identified by the identification information 131 will cancel the service, and is a value calculated by the cancellation prediction device 500 using the prediction model 521 (see FIG. 2 ). The correct answer 133 indicates whether the subscriber has canceled or continued the service, and is a correct answer included in the predicted value data 570.

[0018] <Data analysis support device: Database of subscriber numbers by predicted value> 5 is a data configuration diagram of the number of subscribers by predicted value database 140 according to this embodiment. The number of subscribers by predicted value database 140 is generated by the data acquisition unit 111, which will be described later, based on the predicted value database 130. The number of subscribers by predicted value database 140 is tabular data, and each row (record) includes columns (attributes) of a predicted value 141, a number of cancellations 142, and a number of renewals 143. The number of cancellations 142 is the number of records (subscribers) in the predicted value database 130 for which the predicted value 132 is within the range of the predicted value 141 and the correct answer 133 is "cancellation." The number of renewals 143 is the number of records in the predicted value database 130 for which the predicted value 132 is within the range of the predicted value 141 and the correct answer 133 is "renewal."

[0019] ≪Accuracy≫ Before explaining the control unit 110, prediction accuracy (precision) will be explained. A true positive is when the predicted value 132 of a subscriber (record in the predicted value database 130) is equal to or greater than the threshold, predicting cancellation, and the correct answer 133 being cancellation, and the prediction of cancellation (positive) is correct. A false negative is when the predicted value 132 of a subscriber is less than the threshold, predicting continuation, and the correct answer 133 being cancellation, and the prediction of continuation (negative) is incorrect.

[0020] A false positive is when the predicted value 132 of the policyholder is equal to or greater than the threshold, predicting cancellation, and the correct answer 133 is continuation, and the prediction of cancellation is incorrect. A true negative is when the predicted value 132 of the policyholder is less than the threshold, predicting continuation, and the correct answer 133 is continuation, and the prediction of continuation is correct.

[0021] Prediction accuracy includes precision, recall, and accuracy. Precision is the probability that a prediction of churn is correct when predicted, and is calculated as precision = true positive / (true positive + false positive). In this formula, true positives are the number of true positive subscribers (the number of records in the prediction value database 130), and false positives are the number of false positive subscribers. The same applies to false positives, true negatives, and false negatives in the formulas for precision and accuracy below.

[0022] Recall is the ratio of predicted churn among churn subscribers, calculated as precision = true positives / (true positives + false negatives). Accuracy is the ratio of correct predictions, calculated as accuracy = (true positives + true negatives) / number of subscribers. In general, there is a trade-off between precision and recall. Increasing the threshold increases precision but decreases recall. Conversely, lowering the threshold increases recall but decreases precision.

[0023] <Data analysis support device: control unit> 3, the description of the control unit 110 will continue. The control unit 110 is configured to include a CPU (Central Processing Unit), and is provided with a data acquisition unit 111, an accuracy optimization unit 112, and an effect optimization unit 113. The data acquisition unit 111 acquires the predicted value data 570 (see FIG. 2) and stores it in the predicted value database 130 (see FIG. 4). Next, the data acquisition unit 111 generates the predicted value-specific number of subscribers database 140 (see FIG. 5) based on the predicted value database 130.

[0024] The accuracy optimization unit 112 obtains the lower limit of either the precision or recall specified by the user. Next, the accuracy optimization unit 112 refers to the predicted value database 130 and changes the threshold to find a threshold (also referred to as an accuracy-optimized threshold) that maximizes the other while satisfying the condition for the lower limit. Details will be explained later with reference to FIG. 6.

[0025] The effect optimization unit 113 obtains the cost (loss) of sales activities to encourage subscribers to continue when a user is predicted to cancel, and the profit (expected profit) to be obtained from the sales activities. Next, the effect optimization unit 113 refers to the predicted value database 130 and finds a threshold (also referred to as an effect optimization threshold) that maximizes the effect of subtracting losses from profits while changing the threshold. Details will be explained with reference to FIG. 8 below.

[0026] <Optimal accuracy threshold calculation process> 6 is a flowchart of the precision optimum threshold calculation process according to this embodiment. The process in which the precision optimization unit 112 calculates the threshold that maximizes the precision (precision rate or recall rate) under set conditions will be described with reference to FIG. In step S11, the accuracy optimization unit 112 acquires the conditions for calculating the threshold value via an accuracy optimization threshold value screen 310 (see FIG. 7) described later.

[0027] 7 is a diagram showing the screen layout of the precision optimum threshold screen 310 according to this embodiment. In area 311, the user sets the lower limit of either the precision or the recall as a condition for calculating the threshold. In FIG. 7, the lower limit of the precision is set to 0.40. The user sets the range and precision (tolerance) for determining the threshold in area 312. In Fig. 7, the optimum threshold is set to be calculated with a precision of 0.10 in the range from 0.00 to 1.00.

[0028] When the user has finished setting the parameters, he or she clicks the "Calculate optimal threshold" button 313. In this way, the accuracy optimization unit 112 accepts the lower limit setting accuracy, which is the prediction accuracy for which the lower limit of the precision or recall is set, and the lower limit value. Returning to FIG. 6, the accuracy optimal threshold calculation process will be continued below.

[0029] In step S12, the accuracy optimization unit 112 repeats the process of step S13 while increasing the value set for accuracy by 0.1 increments from the minimum threshold value of 0.0 to the maximum threshold value of 1.0 set in the region 312. In other words, the accuracy optimization unit 112 repeats the process of step S13 while changing the threshold value from 0.0, 0.1, 0.2, ..., 1.0. In step S13, the precision optimization unit 112 calculates the precision rate, recall rate, and accuracy rate at the threshold value.

[0030] In step S14, the accuracy optimization unit 112 specifies a threshold (accuracy-optimized threshold) that satisfies the conditions acquired in step S11 and maximizes the precision or recall rate among the accuracies calculated in step S13. In other words, the accuracy optimization unit 112 specifies a threshold that maximizes the precision rate of 0.4 or more and the recall rate. In step S15, the accuracy optimization unit 112 displays the threshold value identified in step S14 in an area 314 of the accuracy optimization threshold value screen 310 (see FIG. 7), and the accuracy at that threshold value in an area 315.

[0031] <Optimal effect threshold calculation process> 8 is a flowchart of the effect optimization threshold calculation process according to this embodiment. The process in which the effect optimization unit 113 calculates the threshold that maximizes the effect of sales activities under set conditions will be described with reference to FIG. In step S31, the effect optimization unit 113 acquires parameters for threshold calculation via an effect calculation parameter screen 330 (see FIG. 9) described later.

[0032] FIG. 9 is a diagram showing the screen layout of the effect calculation parameter screen 330 according to this embodiment. In the salesperson unit cost field, the user sets the time cost of the salesperson who will engage in sales activities to encourage contract holders predicted to cancel to continue. In the sales man-hour field, the user sets the time (average time) spent on sales activities per contract holder. In FIG. 9, the salesperson unit cost is set at 7,000 yen / hour, the sales man-hours at 4 hours / contract holder, and the cost of sales activities (loss unit) per contract holder (for one prediction target) is 28,000 yen.

[0033] In the "churn prevention success rate" column, the user sets the probability that sales activities will be successful and result in a continuation. In the "churn prevention profit" column, the user sets the profit that will be obtained if the customer continues. In Figure 9, the churn prevention success rate is set to 11%, and the profit from preventing churn per customer is set to 800,000 yen. The profit from preventing churn is determined by factors such as the contract period and contract content, and varies from customer to customer. In such cases, the expected profit (average value) is set as the profit per customer. The profit from sales activities per customer (profit unit), including the success rate, is 88,000 yen. Returning to Figure 8 below, we will continue explaining the effectiveness-optimizing threshold calculation process.

[0034] In step S32, the effect optimization unit 113 repeats steps S33 to S34, starting from the precision optimum threshold (see step S15 in FIG. 6 and area 314 in FIG. 7) and increasing the precision (see area 312) by the value set therefor. According to FIG. 7, the effect optimization unit 113 repeats steps S33 to S34, starting from the precision optimum threshold of 0.7 and increasing it by 0.1. In step S33, the effect optimization unit 113 calculates the effect of changing the threshold from the precision-optimized threshold. In the following description, the precision-optimized threshold is 0.7 and the threshold is 0.8.

[0035] The number of subscribers with a predicted value between 0.7 and 0.8 is 300 + 800 = 1,100 (see Figure 5). For these 1,100 subscribers, changing the threshold from 0.7 to 0.8 changes the prediction result using prediction model 510 from cancellation to continuation, and no sales activities are carried out. This results in a cost reduction (profit) of 28,000 yen x 1,100 = 30.8 million yen.

[0036] The number of churners with a predicted value between 0.7 and 0.8 is 300. By changing the threshold from 0.7 to 0.8, the prediction result for these 300 subscribers changes from churn to renewal, so they are not targeted for sales activities. As a result, if sales activities had been carried out, there would have been an 11% probability of renewal, and profits of 800,000 yen x 300 x 0.11 = 26.4 million yen would have been lost (loss). The effect is 4.4 million yen, calculated by subtracting the reduced cost (profit) of 30.8 million yen from the lost profit (loss) of 26.4 million yen.

[0037] In other words, the effect optimization unit 113 adds the threshold from the precision optimum threshold (0.7) by the allowable error (precision, 0.1) and calculates the positive effect (profit, 30.8 million yen) which is the product of the number of prediction targets (number of subscribers, 1,100) whose prediction values are between the precision optimum threshold (0.7) and the threshold after addition (0.8) and the loss unit, and calculates the negative effect (loss, 26.4 million yen) which is the product of the number of prediction targets (number of cancellations, 300) whose prediction values are between the precision optimum threshold (0.7) and the threshold after addition (0.8) and the correct answer 133 is correct (cancelled) and the profit unit, and subtracts the negative effect from the positive effect to calculate the total effect (effect, 4.4 million yen) when the threshold is increased.

[0038] In step S34, if the effect calculated in step S33 is greater than 0 (step S34→YES), the effect optimization unit 113 increases the threshold by the value of the accuracy and returns to step S33, and if the effect is less than or equal to 0 (step S34→NO), the effect optimization unit 113 proceeds to step S35. In step S35, the effect optimization unit 113 repeats steps S36 to S37 while decreasing the precision by the value set for the precision, starting from the precision optimum threshold value. In step S36, the effect optimization unit 113 calculates the effect of changing the threshold from the precision-optimized threshold. In the following description, the precision-optimized threshold is 0.7 and the threshold is 0.6.

[0039] The number of churners with a predicted value between 0.6 and 0.7 is 450. By changing the threshold from 0.7 to 0.6, the prediction result using prediction model 510 changes from continuation to cancellation, and sales activities will be conducted for these 450 subscribers. If the sales activities are successful with an 11% probability of obtaining continuation, a profit of 800,000 yen x 450 x 0.11 = 39.6 million yen will be obtained.

[0040] The number of subscribers with a predicted value between 0.6 and 0.7 is 450 + 1000 = 1450. By changing the threshold from 0.7 to 0.6, the prediction result using prediction model 510 changes from continuation to cancellation, and sales activities will be conducted toward these 1450 subscribers. This results in a loss of 28,000 yen x 1450 = 40.6 million yen. The effect is -1 million yen, which is the profit of 39.6 million yen minus the loss of 40.6 million yen.

[0041] In other words, the effect optimization unit 113 subtracts the threshold from the precision optimum threshold (0.7) by the allowable error (precision, 0.1) at a time, and calculates the positive effect (profit, 39.6 million yen) which is the product of the number of prediction targets (number of cancellations, 450) whose prediction values are between the precision optimum threshold (0.7) and the threshold after subtraction (0.6) and the correct answer 133 is correct (cancelled) and the profit unit, and calculates the negative effect (loss, 40.6 million yen) which is the product of the number of prediction targets (number of subscribers, 1,450) whose prediction values are between the precision optimum threshold (0.7) and the threshold after subtraction (0.6) and the loss unit, and subtracts the negative effect from the positive effect to calculate the total effect (effect, -1 million yen) when the threshold is reduced.

[0042] In step S37, if the effect calculated in step S36 is greater than 0 (step S37 → YES), the effect optimization unit 113 reduces the threshold by the value of the accuracy and returns to step S36, and if the effect is less than or equal to 0 (step S37 → NO), the effect optimization unit 113 proceeds to step S38. In step S38, the effect optimization unit 113 outputs an improvement effect screen 350 (see FIG. 10) which will be described later. 10 is a diagram showing the screen layout of an improvement effect screen 350 according to this embodiment. The improvement effect screen 350 displays the accuracy and effect when the threshold is changed. The effect (see the improvement effect column in FIG. 10) is maximum when the threshold is 0.8, and 0.8 is the optimum threshold for effect.

[0043] <Features of the data analysis support device> A user of the data analysis support device 100 can obtain a threshold value for the predicted value that maximizes the precision or recall of the prediction model 521 while satisfying the lower limit of either the precision or recall. By setting the precision or recall required to achieve a business performance target, the user can determine the threshold value that maximizes the other. Even users without knowledge of machine learning or data science can set the optimal threshold value for the prediction model according to their intended use.

[0044] Users can also set the profit and loss of sales activities for contract holders predicted to cancel as a response to the prediction results, and obtain the threshold that maximizes the effect of subtracting losses from profits.Users can set the threshold for the prediction value that maximizes the effectiveness / efficiency of their business, not just from the perspective of prediction accuracy.

[0045] <Application example: Failure prediction> In the above embodiment, the data analysis support device 100 has been described using the prediction of an event such as the cancellation / renewal of a service contract as a subject. The prediction of other events is also possible. For example, predicted value data for the prediction of an event such as equipment / machine failure may be used as input data. Precision is important because a large number of false alarms predicting failure when the equipment is normal will increase unnecessary inspections. By using the data analysis support device 100, it becomes possible to set a threshold value that maximizes the recall rate while satisfying the lower limit of precision rate (e.g., 0.8).

[0046] It is also possible to calculate the effect and set a threshold. In the case of failure prediction, the response to equipment / machine predicted to fail is to inspect and repair the equipment / machine. The cost of the response is the cost of maintenance personnel for inspection and repair and the cost of replacement parts. The benefit of the response is the avoidance of losses due to the equipment / machine being stopped, and the benefit from the equipment / machine continuing to operate. It is possible to set a threshold that emphasizes effect based on the benefit and cost (loss) of the countermeasure, inspection and repair.

[0047] <Application example: disease diagnosis> The data analysis support device 100 can also be applied to setting thresholds for devices that diagnose (predict) illnesses from test data. In disease diagnosis, it is important to avoid overlooking illnesses, and recall is therefore important. By using the data analysis support device 100, it becomes possible to set a threshold that maximizes precision while satisfying the lower limit of recall (e.g., 0.9).

[0048] <<Modification: Precision Optimum Threshold Calculation Process>> In the above embodiment, the precision and recall are calculated while increasing the threshold value from the minimum value to the maximum value by the amount of precision, and an optimum precision threshold is found, which is the threshold value that maximizes the recall while satisfying the lower limit of the precision. The threshold may also be calculated while decreasing the threshold value from the maximum value to the minimum value by the amount of precision.

[0049] Since precision and recall are in a trade-off relationship, it is thought that recall will be maximized at the threshold where precision is at its lower limit. Furthermore, since precision also increases as the threshold increases, it is also possible to start from a threshold interval with the minimum threshold where precision is below the lower limit as the lower end and the maximum threshold where precision is above the lower limit as the upper end, and use a bisection method to calculate the threshold where precision is at its lower limit, and use this as the precision-optimized threshold. The same applies when the lower limit of recall is a condition.

[0050] <<Modification: Effect Optimum Threshold Calculation Process>> In the above embodiment, the effect (profit minus loss) is calculated while changing the threshold from the precision-optimal threshold by the amount of precision, and the effect-optimal threshold, which is the threshold that maximizes the effect, is determined. The effect-optimal threshold may be calculated as an optimization problem that maximizes the effect, using the threshold as a variable. For example, a linear programming method may be used to maximize the effect, using the predicted value and the correct answer as constants, and variables indicating the threshold and the cancellation / renewal of each subscriber at that threshold. The profit (reduced cost) is the product of the number of subscribers below the threshold and the sales activity cost per subscriber of 28,000 yen. The loss is the number of cancellations below the threshold and the expected profit of 88,000 yen per subscriber due to preventing cancellations. Other techniques may be used. Generally, as the threshold increases, the effect increases monotonically, reaches a maximum, and then decreases monotonically. Therefore, a ternary search may be used to find the threshold at which the effect is greatest.

[0051] Other variations Although several embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. For example, when calculating the precision optimum threshold, the effect optimization unit 113 uses the same precision as in the precision optimum threshold calculation process (see steps S32 and S35 in FIG. 8), but may acquire a new precision for the effect optimum threshold calculation process via the effect calculation parameter screen 330 (see FIG. 9).

[0052] The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications are included in the scope and spirit of the invention described in this specification, etc., and are also included in the invention described in the claims and their equivalents. [Explanation of symbols]

[0053] 100 Data analysis support device (analysis support device) 112 Precision Optimization Unit 113 Effect Optimization Department 130 Prediction Value Database (Prediction Value Data) 131 Identification Information 132 predicted values 133 Correct

Claims

1. accessing prediction value data in which a prediction value indicating the probability of an event related to the prediction target occurring is associated with a correct answer indicating whether the event related to the prediction target is correct; receiving a lower limit setting accuracy, which is a prediction accuracy for which a lower limit is set between the precision and recall in a case where a prediction value of the prediction target is equal to or greater than a threshold value and a lower limit value; The precision optimization unit calculates an optimum precision threshold, which is the threshold at which the lower limit setting precision is equal to or greater than the lower limit, and which maximizes the precision and recall of the precision and recall that are prediction precisions different from the lower limit setting precision. Data analysis support device.

2. The accuracy optimization unit Accept a setting value of the tolerance of the precision optimum threshold; calculating precision and recall while varying the threshold by the allowable error; The threshold value at which the lower limit value setting accuracy is equal to or greater than the lower limit value and the optimization target accuracy is maximized is identified, and the accuracy optimum threshold value is calculated. The data analysis support device according to claim 1 .

3. Further equipped with an effect optimization section, The effect optimization part is Accepting a tolerance for the accuracy optimum threshold, a loss unit of a response to the event for one prediction target predicted to cause the event, and a profit unit of the response; While increasing the threshold from the precision-optimal threshold by the allowable error, calculate a positive effect, which is the product of the number of prediction objects whose prediction values are between the precision-optimal threshold and the threshold after addition and the loss unit, calculate a negative effect, which is the product of the number of prediction objects whose prediction values are between the precision-optimal threshold and the threshold after addition and for which the correct answer is correct and the profit unit, and calculate a total effect when the threshold is increased by subtracting the negative effect from the positive effect; While subtracting the threshold from the precision-optimal threshold by the allowable error, calculate a positive effect, which is the product of the number of prediction targets whose predicted values are between the precision-optimal threshold and the threshold after the subtraction and whose correct answer is correct, and the profit unit; calculate a negative effect, which is the product of the number of prediction targets whose predicted values are between the precision-optimal threshold and the threshold after the subtraction and the loss unit; and calculate a total effect when the threshold is reduced by subtracting the negative effect from the positive effect; Calculate the optimum effect threshold, which is the threshold at which the maximum total effect is obtained between the total effect when the threshold is increased and the total effect when the threshold is decreased. The data analysis support device according to claim 1 .

4. an effect optimization unit that accesses predicted value data in which a predicted value indicating the probability of an event related to a prediction target occurring is associated with a correct answer indicating whether the event related to the prediction target is correct; The effect optimization unit If the predicted value is equal to or greater than a threshold, a loss unit of a response to the event and a profit unit of the response are received for one prediction target for which the event is predicted to occur; A positive effect is calculated as the product of the number of prediction targets whose predicted value is less than the threshold and the loss unit, a negative effect is calculated as the product of the number of prediction targets whose predicted value is less than the threshold and whose correct answer is correct and the profit unit, and an optimal effect threshold is calculated as the threshold at which the effect obtained by subtracting the negative effect from the positive effect is maximized. Data analysis support device.

5. The effect optimization unit Accept a setting value of the tolerance for the effect optimum threshold; The optimum threshold for effect is calculated while changing the threshold by the allowable error. The data analysis support device according to claim 4.

6. the prediction target is a subscriber of a service, The event is the cancellation of the subscriber's service; The loss unit of the response is a cost required for a response to encourage a subscriber who is predicted to cancel the service to continue the service, The profit unit from the response is the expected profit from the continuation of the service.

5. The data analysis support device according to claim 3.

7. the prediction target is a device, the event is a failure of the device, The response loss unit is a cost required for response to maintain the equipment predicted to fail, The profit unit from the response is the expected value of the profit from preventing the failure.

5. The data analysis support device according to claim 3.

8. The data analysis support device accessing prediction value data in which a prediction value indicating the probability of an event related to the prediction target occurring is associated with a correct answer indicating whether the event related to the prediction target is correct; a step of receiving a lower limit setting accuracy, which is a prediction accuracy for which a lower limit is set between the precision and recall in a case where a prediction value of the prediction target is equal to or greater than a threshold value and the event is predicted to occur, and the lower limit; and calculating an optimum accuracy threshold, which is the threshold at which the lower limit setting accuracy is equal to or greater than the lower limit and which maximizes the precision and recall of the target accuracy to be optimized, which is a prediction accuracy different from the lower limit setting accuracy. Data analysis support methods.

Images