Training program for machine learning model, training method for machine learning model, and training apparatus for machine learning model

Abstract

It is possible to present an improvement proposal in consideration of the possibility of realization while suppressing a decrease in the prediction accuracy of the machine learning model.SOLUTION: A training program for a machine learning model that outputs a proposal for obtaining a desired result, the training program causing a computer to execute processing of acquiring training data including a plurality of attributes, acquiring constraint condition data of the attributes, calculating first information related to prediction accuracy of the machine learning model based on the training data, calculating second information related to feasibility of the proposal based on the training data and the constraint condition data, calculating an evaluation index based on the first information and the second information, and training the machine learning model based on the evaluation index.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a machine learning model training program, a machine learning model training method, and a machine learning model training device. [Background technology]

[0002] Using the training data, the relationship between explanatory variables and dependent variables is modeled, and a machine learning model is trained that outputs the dependent variable result according to the input explanatory variables. For example, in a machine learning model for loan screening, the values ​​of attributes such as the user's age and income are used as explanatory variables, and information about creditworthiness is used as the dependent variable, and the relationship between the explanatory variables and dependent variables is modeled.

[0003] However, when explanatory variables are input into a machine learning model, it is possible that the desired result for the objective variable will not be obtained. To address this situation, there is a technology that presents improvement suggestions to achieve the desired result. For example, consider the case where explanatory variables are input into a machine learning model for loan screening, and the output is an undesired result of the loan screening being denied. In order to achieve the desired result of the loan screening being approved, improvement suggestions (actions), such as increasing income, are presented. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-77904 [Patent Document 2] Japanese Patent Application Publication No. 2023-113928 [Non-patent literature]

[0005] [Non-Patent Document 1] "Learning Models for Actionable Recourse," [Retrieved June 23, 2024], Internet<URL:https: / / openreview.net / pdf?id=JZK9uP4Fev> Summary of the Invention [Problem to be solved by the invention]

[0006] However, with conventional technologies, improvement proposals for achieving desired results may be impossible to implement. For example, when inputting explanatory variables into a machine learning model for loan screening and presenting improvement proposals for achieving desired results, an unrealizable improvement proposal, such as making the age younger, may be presented. Conventional technologies have had the problem of making it difficult to simultaneously consider the prediction accuracy of the machine learning model and the feasibility of the improvement proposal.

[0007] According to one aspect, the present invention aims to provide a technology that can present improvement proposals that take into account feasibility while suppressing a decline in the predictive accuracy of machine learning models. [Means for solving the problem]

[0008] In one aspect, a training program for a machine learning model that outputs proposals for achieving a desired result causes a computer to execute the following processes: acquire training data including a plurality of attributes; acquire constraint data for the attributes; calculate first information regarding the predictive accuracy of the machine learning model based on the training data; calculate second information regarding the feasibility of the proposal based on the training data and the constraint data; calculate an evaluation index based on the first information and the second information; and train the machine learning model based on the evaluation index. [Effects of the Invention]

[0009] According to one aspect, it is possible to present improvement proposals that take into account feasibility while suppressing a decline in the predictive accuracy of machine learning models. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a functional block diagram illustrating a configuration of an information processing device. [Figure 2] FIG. 10 is a diagram illustrating an example of training data. [Figure 3] FIG. 10 is a diagram illustrating an example of constraint data. [Figure 4] FIG. 10 is a diagram illustrating the distribution of training data. [Figure 5] FIG. 10 is a diagram illustrating division of the distribution of training data. [Figure 6] FIG. 10 is a diagram illustrating the distribution of divided training data. [Figure 7] FIG. 1 is a diagram illustrating the application of the present invention to a random forest. [Figure 8] FIG. 1 is a diagram illustrating the application of the present invention to gradient boosting. [Figure 9] FIG. 10 is a diagram illustrating an example of target data. [Figure 10] FIG. 1 is a diagram for explaining a problem that occurs when target data is input into a machine learning model of the prior art. [Figure 11] FIG. 10 is a diagram for explaining the effect when target data is input into the machine learning model of the present embodiment. [Figure 12] 10 is a flowchart illustrating a flow of processing by an information processing device. [Figure 13] 1 is a flowchart illustrating a detailed example process for training a machine learning model. [Figure 14] 1 is an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0011] A machine learning model training program, a machine learning model training method, and a machine learning model training device according to embodiments will be described below with reference to the accompanying drawings. Note that the disclosed technology is not limited to these embodiments. The embodiments can be appropriately combined within the scope of not causing any contradiction in the processing content.

[0012] 1 is a functional block diagram illustrating the configuration of an information processing device (machine learning model training device) according to this embodiment. As shown in FIG. 1, the information processing device 1 includes a communication unit 110, an input unit 120, an output unit 130, a storage unit 140, and a control unit 150.

[0013] The communication unit 110 is a processing unit that executes data communication with an external device (not shown) via a network. The communication unit 110 is an example of a communication device. The information processing device 1 may acquire training data and constraint condition data, which will be described later, from the external device. The information processing device 1 may also acquire target data, which will be described later, from the external device.

[0014] The input unit 120 is an input device for inputting various types of information to the information processing device 1. A user may operate the input unit 120 to input training data and constraint condition data, which will be described later. A user may also operate the input unit 120 to input target data, which will be described later. The output unit 130 is an output device that displays information output from the control unit 150.

[0015] The storage unit 140 is a functional unit that stores various types of information that are acquired, referenced, etc. in the information processing device 1, including an OS (Operating System) executed by the control unit 150. The storage unit 140 stores training data 141, constraint condition data 142, evaluation indexes 143, and machine learning models 144. The control unit 150 is a functional unit that performs overall control of the information processing device 1. The control unit 150 has an acquisition unit 151, a calculation unit 152, a training unit 153, and an inference unit 154.

[0016] The acquisition unit 151 acquires, from the communication unit 110 or the input unit 120, a plurality of training data (learning data) used to train (learn) the machine learning model, and constraint condition data.

[0017] Fig. 2 is a diagram showing an example of training data. As shown in Fig. 2, the training data 200 associates attributes with attribute values. In the example of Fig. 2, the attribute and attribute value are set as explanatory variables, such that "monthly income is 360,000 yen, overtime hours are 20 hours, and frequency of business trips is 4 times." Furthermore, the attribute and attribute value are set as objective variables, such that "credit risk is high."

[0018] 2 shows one training data set indicating that a user with a monthly income of 360,000 yen, 20 hours of overtime work, and four business trips per day is a high credit risk in loan screening. The acquisition unit 151 acquires a plurality of training data sets related to a plurality of users, or a training data set including the plurality of training data sets. The storage unit 140 stores training data 141 related to the acquired plurality of training data sets or training data set.

[0019] In FIG. 2, monthly income, overtime hours, frequency of business trips, etc. are exemplified as attributes, but they are not limited to these. For example, attributes may be any conceivable attribute, such as loan purpose, educational background, number of outstanding loans, etc. Also, in FIG. 2, a value of 360,000 yen is exemplified as a monthly income value, but they are not limited to this. Furthermore, the monthly income value may be within a certain range or condition, such as 400,000 yen or more, or 400,000 yen to less than 500,000 yen. The same applies to values ​​of other attributes.

[0020] Fig. 3 is a diagram showing an example of constraint condition data. As shown in Fig. 3, the constraint condition data 300 associates attributes with attribute constraint conditions. In the example of Fig. 3, the attributes and attribute constraint conditions are set as follows: "monthly salary can be changed, overtime hours can only be reduced, business trip frequency can be changed, and age cannot be changed."

[0021] FIG. 3 shows that, with regard to improvement proposals for obtaining a desired result in loan screening, monthly income can be changed, overtime hours can only be reduced, business trip frequency can be changed, and age cannot be changed. Note that attribute constraints can be changed as appropriate. The acquisition unit 151 acquires constraints for each of the multiple attributes. The storage unit 140 stores constraint data 142 related to the constraints for each of the multiple acquired attributes.

[0022] In FIG. 3, as in FIG. 2, monthly income, overtime hours, frequency of business trips, etc. are exemplified as attributes, but they are not limited to these. For example, attributes may be any conceivable attribute, such as loan purpose, educational background, number of outstanding loans, etc. Also, in FIG. 3, constraints on monthly income are exemplified as being changeable, but they are not limited to these. Also, the constraints on monthly income may be unchangeable, or may be specific conditions, such as only allowing an increase up to 30,000 yen. The same applies to constraints on other attributes.

[0023] Returning to the explanation of FIG. 1, the calculation unit 152 calculates an evaluation index based on first information related to the prediction accuracy of the machine learning model and second information related to the feasibility of the improvement proposal. The calculation unit 152 may calculate the first information based on training data. The calculation unit 152 may calculate the second information based on the training data and constraint condition data. The storage unit 140 stores an evaluation index 143 related to the calculated evaluation index. Note that the storage unit 140 may store the first information and the second information.

[0024] The training unit 153 trains the machine learning model based on the training data and the constraint condition data. In this embodiment, the machine learning model is described as a decision tree model, but this is not limiting. The training unit 153 trains the machine learning model based on the calculated evaluation index. The training unit 153 may recursively train the machine learning model. Calculation of the evaluation index and training of the machine learning model will be described in detail with reference to FIGS. 4 to 6. Note that the function of the training unit 153 may include the function of the calculation unit 152.

[0025] FIG. 4 is a diagram illustrating the distribution of training data. Graph 400 in FIG. 4 shows the distribution of multiple training data, with the vertical and horizontal axes corresponding to the attributes of the training data, and the data represented by triangles and circles corresponding to the attribute values ​​of the training data. For convenience of explanation, FIG. 4 illustrates a two-dimensional graph with vertical and horizontal axes for two attributes, but this is not limited to this. For example, an N-dimensional graph with N axes for N attributes (N is a natural number greater than or equal to 3) may be used. Also, while FIG. 4 illustrates six data, this is not limited to this.

[0026] In Figure 4, the vertical axis corresponds to the attribute of the number of outstanding loans, and the horizontal axis corresponds to the attribute of age. Here, the constraint on the number of outstanding loans can be changed, but the constraint on age cannot be changed. Data marked with triangles are data with labels (objective variables) that result in the loan being rejected, and data marked with circles are data with labels that result in the loan being approved.

[0027] In FIG. 4, data 40a, data 40d, and data 40e are data labeled with a loan approval, and data 40b, data 40c, and data 40f are data labeled with a loan denial.

[0028] FIG. 5 is a diagram illustrating the division of the distribution of training data. The calculation unit 152 divides the training data into multiple regions, including a first region and a second region, using a division pattern of multiple dividing lines. The calculation unit 152 divides and predicts labels based on the first region and the second region. FIG. 5 illustrates three first division patterns for dividing the training data. FIG. 5(a) illustrates graph 410, which is first division pattern 1 of graph 400. FIG. 5(b) illustrates graph 420, which is first division pattern 2 of graph 400. FIG. 5(c) illustrates graph 430, which is first division pattern 3 of graph 400.

[0029] Graph 410 in Fig. 5(a) is a diagram in which data 40a and data 40d, and data 40b and data 40e are divided by a dividing line 41 parallel to the vertical axis. The dividing line 41 divides the data into areas 41a on the left side of the dividing line 41 where the loan is predicted to be approved and areas 41b on the right side of the dividing line 41 where the loan is predicted to be rejected.

[0030] Here, the prediction accuracy based on the division line 41 will be described. The prediction accuracy based on the division line 41 is an example of the first information. The calculation unit 152 calculates the first information based on the number of training data items that match the labels of the first and second regions. As shown in FIG. 5(a), Of the data included in area 41a where the loan is predicted to be approved, there are two pieces of data labeled "loan approved": data 40a and data 40d. Furthermore, of the data included in area 41b where the loan is predicted to be rejected, there are three pieces of data labeled "loan rejected": data 40b, data 40c, and data 40f. In other words, of the total six pieces of data, five pieces of data match the prediction that the loan will be approved or rejected. Therefore, in the example shown in FIG. 5(a), calculation unit 152 calculates the prediction accuracy as 5 / 6.

[0031] Next, the degree of assurance of an improvement proposal based on the dividing line 41 will be described. The degree of assurance of an improvement proposal based on the dividing line 41 is an example of second information. The calculation unit 152 calculates the second information based on the number of training data items for which a change from the first region to the second region can be proposed. As illustrated in FIG. 5(a), the region 41a predicting that the loan will be approved contains two pieces of data: data 40a and data 40d. The region 41b predicting that the loan will be rejected contains four pieces of data: data 40b, data 40c, data 40e, and data 40f. Now, consider a case where an improvement proposal is made. When making an improvement proposal to change data labeled "rejection of the loan," which is not the desired result, to "approval of the loan," which is the desired result, it is necessary to consider whether the four pieces of data included in the region 41b can be moved to the region 41a. However, age, which is the horizontal axis of the graph 410, cannot be changed due to constraints. Therefore, the four pieces of data included in the region 41b cannot be moved to the region 41a.

[0032] 5(a), the calculation unit 152 calculates the degree of assurance of the improvement proposal as 2 / 6 by including only two pieces of data included in the area 41a out of a total of six pieces of data. Note that the calculation unit 152 may calculate the degree of assurance of the improvement proposal as 0 / 6 without including the two pieces of data included in the area 41a.

[0033] Next, the evaluation index based on the dividing line 41 will be described. The calculation unit 152 calculates the evaluation index by adding the prediction accuracy and the assurance level of the improvement proposal. In the example shown in Fig. 5(a), the prediction accuracy of 5 / 6 is added to the assurance level of the improvement proposal of 2 / 6, and the evaluation index is calculated as 7 / 6.

[0034] Graph 420 in FIG. 5(b) is a diagram in which data 40b and data 40e, and data 40c and data 40f are divided by dividing line 42 parallel to the vertical axis. Dividing line 42 divides the data, with region 42a on the left side of dividing line 42 predicting that the loan will be approved and region 42b on the right side of dividing line 42 predicting that the loan will be rejected.

[0035] Here, the prediction accuracy based on the division line 42 will be described. The prediction accuracy based on the division line 42 is an example of first information. The calculation unit 152 calculates the first information based on the number of training data items that match the labels of the first and second regions. As illustrated in FIG. 5(b), among the data items included in the region 42a where the loan is predicted to be approved, three pieces of data have the label "loan approved": data 40a, data 40d, and data 40e. Furthermore, among the data items included in the region 42b where the loan is predicted to be rejected, two pieces of data have the label "loan rejected": data 40c and data 40f. In other words, five pieces of data out of a total of six pieces of data match the prediction that the loan will be approved or rejected. Therefore, in the example shown in FIG. 5(b), the calculation unit 152 calculates the prediction accuracy as 5 / 6.

[0036] Next, the degree of assurance of an improvement proposal based on the dividing line 42 will be described. The degree of assurance of an improvement proposal based on the dividing line 42 is an example of second information. The calculation unit 152 calculates the second information based on the number of training data items for which a change from the first region to the second region can be proposed. As illustrated in FIG. 5(b), the region 42a predicting that the loan will be approved contains four pieces of data: data 40a, data 40b, data 40d, and data 40e. The region 42b predicting that the loan will be rejected contains two pieces of data: data 40c and data 40f. Now, consider a case where an improvement proposal is made. When making an improvement proposal to change data labeled as a loan rejection, which is not the desired result, to a loan approval, which is the desired result, it is necessary to consider whether the two pieces of data included in the region 42b can be moved to the region 42a. However, age, which is the horizontal axis of the graph 420, cannot be changed due to constraints. Therefore, the two pieces of data included in the region 42b cannot be moved to the region 42a.

[0037] 5(b), the calculation unit 152 calculates the degree of assurance of the improvement proposal as 4 / 6 by including only the four pieces of data included in the area 42a out of a total of six pieces of data. Note that the calculation unit 152 may calculate the degree of assurance of the improvement proposal as 0 / 6 without including the four pieces of data included in the area 42a.

[0038] Next, the evaluation index based on the dividing line 42 will be described. The calculation unit 152 calculates the evaluation index by adding the prediction accuracy and the assurance level of the improvement proposal. In the example shown in Fig. 5(b), the prediction accuracy of 5 / 6 is added to the assurance level of the improvement proposal of 4 / 6, and the evaluation index is calculated as 9 / 6.

[0039] Graph 430 in Figure 5(c) is a diagram in which data 40a, data 40b, and data 40c, and data 40d, data 40e, and data 40f are divided by a dividing line 43 parallel to the horizontal axis. The dividing line 43 divides the data into areas 43a below the dividing line 43 where the loan is predicted to be approved and areas 43b above the dividing line 43 where the loan is predicted to be rejected.

[0040] Here, the prediction accuracy according to the division line 43 will be described. The prediction accuracy according to the division line 43 is an example of first information. The calculation unit 152 calculates the first information based on the number of training data items that match the labels of the first and second regions. As illustrated in FIG. 5(c), among the data items included in the region 43a where the loan is predicted to be approved, there are two pieces of data labeled as "approved": data 40d and data 40e. Furthermore, among the data items included in the region 43b where the loan is predicted to be rejected, there are two pieces of data labeled as "rejected": data 40b and data 40c. In other words, among the six pieces of data in total, four pieces of data match the prediction that the loan will be approved or rejected. Therefore, in the example shown in FIG. 5(c), the calculation unit 152 calculates the prediction accuracy as 4 / 6.

[0041] Next, the degree of guarantee of the improvement proposal based on the dividing line 43 will be described. The degree of guarantee of the improvement proposal based on the dividing line 43 is an example of second information. The calculation unit 152 calculates the second information based on the number of training data items that can be used to propose a change from the first region to the second region. As illustrated in FIG. 5(c), the region 43a predicting that the loan will be approved contains three pieces of data: data 40d, data 40e, and data 40f. The region 43b predicting that the loan will be rejected contains three pieces of data: data 40a, data 40b, and data 40c. Now, consider the case where an improvement proposal is made. When making an improvement proposal to change data labeled as loan rejection, which is not the desired result, to loan approval, which is the desired result, it is necessary to consider whether the three pieces of data included in the region 43b can be moved to the region 43a. The number of unpaid loans, which is the vertical axis of the graph 430, can be changed using constraints. Therefore, the three pieces of data included in the region 43b can be moved to the region 43a.

[0042] 5(c), the calculation unit 152 calculates the degree of assurance of the improvement proposal as 6 / 6 by including three pieces of data included in the area 43a and three pieces of data included in the area 43b out of a total of six pieces of data. Note that the calculation unit 152 may calculate the degree of assurance of the improvement proposal as 3 / 6 without including the three pieces of data included in the area 43a.

[0043] Next, the evaluation index based on the dividing line 43 will be described. The calculation unit 152 calculates the evaluation index by adding the prediction accuracy and the assurance level of the improvement proposal. In the example shown in Fig. 5(c), the prediction accuracy of 4 / 6 is added to the assurance level of the improvement proposal of 6 / 6, and the evaluation index is calculated as 10 / 6.

[0044] Although the dividing lines 41, 42, and 43 of this embodiment are described as being parallel to the horizontal or vertical axis, they are not limited to this. For example, the dividing lines may be oblique to the horizontal or vertical axis.

[0045] 5, the dividing lines 41, 42, and 43 of this embodiment are shown as straight lines, but they are not limited to these. For example, the dividing lines may be curved, broken lines, or a combination of straight lines, curved lines, and broken lines.

[0046] The prediction accuracy, the assurance level of improvement proposals, and the evaluation index calculated by the calculation unit 152 in this embodiment may not be limited to the calculation methods described above. For example, although the calculation unit 152 divides the prediction accuracy and the assurance level of improvement proposals by the total number of data, they do not have to be divided by the total number of data.

[0047] Alternatively, the calculation unit 152 may multiply the prediction accuracy and the degree of assurance of the improvement proposal by a coefficient, and add the prediction accuracy and the degree of assurance of the improvement proposal multiplied by the coefficient to calculate the evaluation index. Alternatively, the calculation unit 152 may calculate the evaluation index by multiplying the prediction accuracy and the degree of assurance of the improvement proposal. The balance between the prediction accuracy and the degree of assurance of the improvement proposal can be adjusted by a hyperparameter.

[0048] The calculation unit 152 may calculate the prediction accuracy, the assurance level of the improvement proposal, and the evaluation index using any calculation method that can express the prediction accuracy, the assurance level of the improvement proposal, and the evaluation index.

[0049] The evaluation index 143 calculated by the calculation unit 152 is stored in the storage unit 140. The prediction accuracy or the assurance level of the improvement proposal calculated by the calculation unit 152 may also be stored in the storage unit 140.

[0050] The training unit 153 selects a division line that will result in the highest evaluation index from among multiple division patterns for the data, and divides the data at the selected division line. In the example shown in Fig. 5, the training unit 153 selects division line 43 that will result in the highest evaluation index from division lines 41, 42, and 43, and divides the graph 400 at the selected division line 43. The training unit 153 selects the division line that will result in the highest evaluation index based on the training data and the constraint condition data, and trains the machine learning model.

[0051] Fig. 6 is a diagram illustrating the distribution of divided training data. Fig. 6 illustrates the distribution of two sets of training data divided by a division line 43 with the highest evaluation index calculated in Fig. 5. Fig. 6(a) illustrates a graph 431 including data 40a, 40b, and 40c located above the division line 43 of graph 430. Fig. 6(b) illustrates a graph 432 including data 40d, 40e, and 40f located below the division line 43 of graph 430.

[0052] The calculation unit 152 or the training unit 153 may perform the same processing as described above for the graph 400 illustrated in Fig. 4 on the graph 431 illustrated in Fig. 6(a) and the graph 432 illustrated in Fig. 6(b). The calculation unit 152 performs processing to divide the distribution of the divided data into a plurality of regions including a third region and a fourth region using a plurality of second division patterns.

[0053] The calculation unit 152 calculates an evaluation index for each of the plurality of second division patterns. The training unit 153 divides the distribution of the divided data by the second division pattern with the highest evaluation index, thereby generating a distribution of further divided data.

[0054] The training unit 153 selects the division line that maximizes the evaluation index based on the training data and the constraint condition data, and recursively trains the machine learning model. Note that the training unit 153 may perform recursive training of the decision tree model a predetermined number of times and then terminate the process in order to prevent overlearning.

[0055] The training technique for the decision tree model of this embodiment can be extended to a training technique for an ensemble model of decision trees. Examples of training techniques for the ensemble model include random forests and gradient boosting. Examples of gradient boosting techniques include XGBoost (Extreme Gradient Boosting) and LightGBM (Light Gradient Boosting Machine).

[0056] 7 is a diagram illustrating the application of the present invention to a random forest. As illustrated in FIG. 7, the decision tree model training technique of the present embodiment can be applied in parallel to multiple bootstrap samples of training data 500.

[0057] 8 is a diagram illustrating the application of the present invention to gradient boosting. As illustrated in FIG. 7, the decision tree model training technique of this embodiment can be applied to training data 600, and multiple decision tree models can be trained sequentially so that the new learning index in the training technique of this embodiment improves.

[0058] A plurality of trained decision tree models may be integrated to form an ensemble model of decision trees. The trained machine learning model 144 is stored in the storage unit 140.

[0059] Returning to the explanation of Figure 1, the inference unit 154 inputs the target data (explanatory variables) to be inferred into a trained machine learning model, and infers whether a desired result can be obtained (objective variable), and if the desired result cannot be obtained, proposes an improvement to obtain the desired result. The output unit 130 outputs the inferred improvement proposal.

[0060] Fig. 9 is a diagram showing an example of target data. As shown in Fig. 9, the target data 700 has attributes and attribute values ​​associated with each other. In the example of Fig. 9, the following explanatory variables are set for the attributes and attribute values: annual income is $70,000, purpose of loan is to purchase a new car, highest level of education is college graduation, and number of outstanding loans is 2.

[0061] In this embodiment, the inference unit 154 inputs target data to be inferred into a trained machine learning model, and if a desired result is not obtained, presents an improvement proposal that takes into account feasibility. For example, if the target data 700 in FIG. 9 is input into the machine learning model and an undesired result of high credit risk is obtained, the inference unit 154 presents an improvement proposal that takes into account feasibility, namely, reducing the number of unpaid loans.

[0062] Here, the effects achieved by the above-described information processing will be described.

[0063] In this embodiment, the information processing device 1 trains a machine learning model based on training data and constraint data. For example, in this embodiment, the information processing device 1 calculates an evaluation index based on first information on the prediction accuracy of the machine learning model and second information on the feasibility of a proposal, and trains the machine learning model based on the calculated evaluation index. This allows the information processing device 1 to generate a machine learning model that can present improvement proposals that take feasibility into consideration while suppressing a decrease in the prediction accuracy of the machine learning model.

[0064] In this embodiment, the information processing device 1 performs a process of dividing the distribution of multiple training data into multiple regions using multiple first division patterns, and calculates an evaluation index for each of the multiple first division patterns. The information processing device 1 divides the distribution of the training data using the first division pattern with the highest evaluation index, thereby generating a distribution of divided training data and training a decision tree model.

[0065] In this embodiment, the information processing device 1 can be applied to training a non-differentiable model such as an ensemble model of decision trees. Prediction tasks where it is important to ensure the existence of improvement suggestions, such as loan screening, are generally in a tabular data format, and this embodiment can be applied even in such cases.

[0066] In this embodiment, the information processing device 1 calculates the first information based on the number of training data items that match the labels of each of the multiple regions, thereby enabling the information processing device 1 to take into account the prediction accuracy of the machine learning model.

[0067] In this embodiment, the information processing device 1 calculates the second information based on the number of training data that can be used to propose a change from the first domain to the second domain, thereby enabling the information processing device 1 to consider the feasibility of the improvement proposal.

[0068] In this embodiment, the information processing device 1 recursively trains the machine learning model, thereby enabling the information processing device 1 to improve the prediction accuracy of the machine learning model and the feasibility of improvement proposals.

[0069] In this example, we have explained the training of a machine learning model that takes into account prediction accuracy and the feasibility of improvement proposals in loan screening, but this does not have to be limited to loan screening and can be applied to training any machine learning model.

[0070] For example, in the case of education, this embodiment may be applied to training a machine learning model that suggests a field to study in order to pass a certain exam. Furthermore, in the case of human resources, this embodiment may be applied to training a machine learning model that suggests skills to acquire in order to be hired by a certain company or department. In this way, this embodiment can be applied to training machine learning models used in various fields and workplaces.

[0071] The effects achieved by this embodiment will be described in detail with reference to FIGS.

[0072] First, the problems with the conventional technology will be explained. Fig. 10 is a diagram for explaining the problems that arise when the target data 700 in Fig. 9 is input into a machine learning model of the conventional technology. As shown by the solid arrow in Fig. 10, the inference result is that the loan is denied because the purpose of the loan is to purchase a new car and the educational background is not a master's degree or higher.

[0073] Now, let us consider the improvement proposals to obtain the desired result of loan approval. As shown in Figure 10, in order for the loan to be approved, it is necessary to change either the educational background or the purpose of the loan, and there is an issue in that only improvement proposals with low feasibility exist.

[0074] Next, the effect of this embodiment will be described. Fig. 11 is a diagram for explaining the effect when the target data 700 of Fig. 9 is input into the machine learning model of this embodiment. As shown by the solid arrow in Fig. 11, the inference result is that the loan is denied not because the annual income is less than $50,000, but because the number of unpaid loans is two or more.

[0075] Now, consider an improvement proposal for obtaining loan approval, which is the desired outcome. As shown in FIG. 11, there is an improvement proposal with a high feasibility, which is to change the number of outstanding loans in order to obtain loan approval. In this way, the information processing device 1 of this embodiment can generate a machine learning model that can present improvement proposals that take feasibility into consideration while suppressing a decrease in the prediction accuracy of the machine learning model.

[0076] FIG. 12 is a flowchart illustrating the flow of processing by the information processing device 1 of this embodiment. As shown in FIG. 12, the acquisition unit 151 acquires training data (step S1). Subsequently, the acquisition unit 151 acquires constraint condition data (step S2). Then, the calculation unit 152 calculates an evaluation index, and the training unit 153 selects a division line that increases the evaluation index and trains multiple machine learning models (decision tree models) (step S3). Finally, the training unit 153 integrates the trained multiple machine learning models (step S4). Note that FIG. 12 illustrates training multiple machine learning models and integrating the trained multiple machine learning models, but this is not limiting. For example, the processing may be terminated by training a single machine learning model.

[0077] FIG. 13 is a flowchart illustrating the details of the process of training a machine learning model. As shown in FIG. 13, the calculation unit 152 lists division line candidates on the training data (step S11). Subsequently, the calculation unit 152 calculates an evaluation index (step S12). Then, the training unit 153 divides the training data using a division line that maximizes the evaluation index (step S13). Next, if the training unit 153 has divided the training data a predetermined number of times (step S14: Yes), the training unit 153 ends the process. If the training data has not been divided a predetermined number of times (step S14: No), the training unit 153 executes the process of step S11 on the divided training data.

[0078] Fig. 14 shows an example of the hardware configuration of the information processing device 1 of this embodiment. As shown in Fig. 14, the information processing device 1 includes a communication device 10a, a hard disk drive (HDD) 10b, a memory 10c, and a processor 10d. The components shown in Fig. 14 are connected to each other via a bus or the like.

[0079] The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and DBs that operate the functions shown in FIG.

[0080] The processor 10d reads out from the HDD 10b or the like a program that executes the same processes as the respective processing units shown in FIG. 1 and loads it into the memory 10c, thereby operating a process that executes each function described in FIG. 1 or the like. For example, this process executes the same functions as the respective processing units of the information processing device 1. Specifically, the processor 10d reads out from the HDD 10b or the like a program that has the same functions as the acquisition unit 151, the calculation unit 152, the training unit 153, and the inference unit 154. Then, the processor 10d executes a process that executes the same processes as the acquisition unit 151, the calculation unit 152, the training unit 153, and the inference unit 154.

[0081] In this way, the information processing device 1 operates as an information processing device that reads and executes a program to train a machine learning model. The information processing device 1 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 1. For example, the present invention can also be applied in the same way to cases where another computer or server executes the program, or where these execute the program in cooperation with each other.

[0082] This program can be distributed via a network such as the Internet. In addition, this program can be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and can be executed by being read from the recording medium by a computer.

[0083] Although the disclosed embodiments and their advantages have been described in detail, those skilled in the art will recognize that various modifications, additions, and omissions may be made thereto without departing from the scope of the invention as clearly set forth in the claims.

[0084] The following additional notes are provided regarding the embodiment described with reference to FIGS. (Appendix 1) A program for training a machine learning model that outputs suggestions for achieving a desired outcome, Obtain training data containing multiple attributes, obtaining constraint data for the attribute; calculating first information regarding the predictive accuracy of the machine learning model based on the training data; calculating second information regarding the feasibility of the proposal based on the training data and the constraint data; Calculating an evaluation index based on the first information and the second information; training the machine learning model based on the evaluation metrics; A machine learning model training program that causes a computer to execute processing. (Appendix 2) the machine learning model is a decision tree model, The process of training the machine learning model includes: performing a process of dividing the distribution of the plurality of training data into a plurality of regions including a first region and a second region using a plurality of first division patterns; Calculating the evaluation index for each of the plurality of first division patterns; Dividing the distribution of the training data using a first division pattern that has the highest evaluation index, thereby generating a distribution of divided training data. 2. A training program for a machine learning model according to claim 1, comprising: (Appendix 3) The process of calculating the first information calculates the first information based on the number of training data items that match the labels of the first region and the second region. 3. A training program for a machine learning model according to claim 2, comprising: (Appendix 4) The process of calculating the second information calculates the second information based on the number of training data that can be used to propose a change from the first domain to the second domain. 3. A training program for a machine learning model according to claim 2, comprising: (Appendix 5) The process of training the machine learning model includes: performing a process of dividing the distribution of the divided training data into a plurality of regions including a third region and a fourth region using a plurality of second division patterns; Calculating the evaluation index for each of the plurality of second division patterns; Dividing the distribution of the divided training data using a second division pattern that has the highest evaluation index, thereby generating a distribution of further divided training data. 3. A training program for a machine learning model according to claim 2, comprising: (Appendix 6) The machine learning model is an ensemble learning model. 6. A training program for a machine learning model according to any one of appendices 2 to 5. (Appendix 7) 1. A method for training a machine learning model that outputs suggestions for achieving a desired outcome, comprising: Obtain training data containing multiple attributes, obtaining constraint data for the attribute; calculating first information regarding the predictive accuracy of the machine learning model based on the training data; calculating second information regarding the feasibility of the proposal based on the training data and the constraint data; Calculating an evaluation index based on the first information and the second information; training the machine learning model based on the evaluation metrics; A method for training a machine learning model, characterized by having a computer execute processing. (Appendix 8) the machine learning model is a decision tree model, The process of training the machine learning model includes: performing a process of dividing the distribution of the plurality of training data into a plurality of regions including a first region and a second region using a plurality of first division patterns; Calculating the evaluation index for each of the plurality of first division patterns; Dividing the distribution of the training data using a first division pattern that has the highest evaluation index, thereby generating a distribution of divided training data. 8. The method for training a machine learning model according to claim 7, further comprising the steps of: (Appendix 9) The process of calculating the first information calculates the first information based on the number of training data items that match the labels of the first region and the second region. 9. A method for training a machine learning model according to claim 8, comprising the steps of: (Appendix 10) The process of calculating the second information calculates the second information based on the number of training data that can be used to propose a change from the first domain to the second domain. 9. A method for training a machine learning model according to claim 8, comprising the steps of: (Appendix 11) The process of training the machine learning model includes: performing a process of dividing the distribution of the divided training data into a plurality of regions including a third region and a fourth region using a plurality of second division patterns; Calculating the evaluation index for each of the plurality of second division patterns; Dividing the distribution of the divided training data using a second division pattern that has the highest evaluation index, thereby generating a distribution of further divided training data. 9. A method for training a machine learning model according to claim 8, comprising the steps of: (Appendix 12) The machine learning model is an ensemble learning model. 12. The method for training a machine learning model according to any one of appendices 8 to 11, (Appendix 13) An apparatus for training a machine learning model that outputs suggestions for achieving a desired result, an acquisition unit that acquires training data including a plurality of attributes and constraint condition data for the attributes; a training unit that calculates first information regarding the prediction accuracy of the machine learning model based on the training data, calculates second information regarding the feasibility of the proposal based on the training data and the constraint condition data, calculates an evaluation index based on the first information and the second information, and trains the machine learning model based on the evaluation index; A training device for machine learning models. (Appendix 14) the machine learning model is a decision tree model, The training department: performing a process of dividing the distribution of the plurality of training data into a plurality of regions including a first region and a second region using a plurality of first division patterns; Calculating the evaluation index for each of the plurality of first division patterns; Dividing the distribution of the training data using a first division pattern that has the highest evaluation index, thereby generating a distribution of divided training data. 14. The machine learning model training device according to claim 13, (Appendix 15) the training unit calculates the first information based on the number of training data items that match the labels of the first region and the second region. 15. The machine learning model training device according to claim 14, (Appendix 16) The training unit calculates the second information based on the number of training data that can be used to propose a change from the first domain to the second domain. 15. The machine learning model training device according to claim 14, (Appendix 17) The training department: performing a process of dividing the distribution of the divided training data into a plurality of regions including a third region and a fourth region using a plurality of second division patterns; Calculating the evaluation index for each of the plurality of second division patterns; Dividing the distribution of the divided training data using a second division pattern that has the highest evaluation index, thereby generating a distribution of further divided training data. 15. The machine learning model training device according to claim 14, (Appendix 18) The machine learning model is an ensemble learning model. 18. The machine learning model training device according to any one of appendices 14 to 17, [Explanation of symbols]

[0085] 1. Information processing equipment 110 Communications Department 120 Input section 130 Output section 140 Storage section 141 training data 142 Constraint Data 143 Evaluation Indicators 144 Machine Learning Models 150 control section 151 Acquisition Department 152 Calculation Unit 153 Training Department 154 Reasoning part

Claims

1. A program for training a machine learning model that outputs suggestions for achieving a desired outcome, Obtain training data containing multiple attributes, obtaining constraint data for the attribute; calculating first information regarding the predictive accuracy of the machine learning model based on the training data; calculating second information regarding the feasibility of the proposal based on the training data and the constraint data; Calculating an evaluation index based on the first information and the second information; training the machine learning model based on the evaluation metrics; A machine learning model training program that causes a computer to execute processing.

2. the machine learning model is a decision tree model, The process of training the machine learning model includes: performing a process of dividing the distribution of the plurality of training data into a plurality of regions including a first region and a second region using a plurality of first division patterns; calculating the evaluation index for each of the plurality of first division patterns; Dividing the distribution of the training data using a first division pattern that has the highest evaluation index, thereby generating a distribution of divided training data. The machine learning model training program according to claim 1, further comprising:

3. the process of calculating the first information calculates the first information based on the number of training data items that match the labels of the first region and the second region, The machine learning model training program according to claim 2, further comprising a process.

4. The process of calculating the second information calculates the second information based on the number of training data that can be used to propose a change from the first domain to the second domain. The machine learning model training program according to claim 2, further comprising a process.

5. The process of training the machine learning model includes: performing a process of dividing the distribution of the divided training data into a plurality of regions including a third region and a fourth region using a plurality of second division patterns; calculating the evaluation index for each of the plurality of second division patterns; Dividing the distribution of the divided training data using a second division pattern that has the highest evaluation index, thereby generating a distribution of further divided training data. The machine learning model training program according to claim 2, further comprising a process.

6. The machine learning model is an ensemble learning model. The training program for a machine learning model according to any one of claims 2 to 5.

7. 1. A method for training a machine learning model that outputs suggestions for achieving a desired outcome, comprising: Obtain training data containing multiple attributes, obtaining constraint data for the attribute; calculating first information regarding the predictive accuracy of the machine learning model based on the training data; calculating second information regarding the feasibility of the proposal based on the training data and the constraint data; Calculating an evaluation index based on the first information and the second information; training the machine learning model based on the evaluation metrics; A method for training a machine learning model, characterized by having a computer execute processing.

8. An apparatus for training a machine learning model that outputs suggestions for achieving a desired result, an acquisition unit that acquires training data including a plurality of attributes and constraint condition data for the attributes; a training unit that calculates first information regarding the prediction accuracy of the machine learning model based on the training data, calculates second information regarding the feasibility of the proposal based on the training data and the constraint condition data, calculates an evaluation index based on the first information and the second information, and trains the machine learning model based on the evaluation index; A training device for machine learning models.

Images