Index selection device, index selection method, and program

Abstract

An indicator selection device (10) comprises an acquisition unit (110) and a selection unit (120). The acquisition unit (110) acquires information (hereinafter described as "category variable specification information") specifying a category variable. The selection unit (120) selects a support variable from a plurality of indicators described above for each category variable. Specifically, the selection unit (120) generates a first model by executing machine learning in which a combination (combination variable) of the category variable and the support variable is set as an explanatory variable and the result of evaluating an evaluation subject is set as an objective variable. A first degree of influence is generated for each of a plurality of combination variables. The first degree of influence indicates the degree of influence on the accuracy of the first model by the combination variable. The selection unit (120) uses the first degree of influence to select a support variable.

Description

[Technical field]

[0001] The present invention relates to an index selection device, an index selection method, and a program. [Background technology]

[0002] In recent years, relationships between objective variables and explanatory variables have been found by analyzing large amounts of data. On the other hand, as described in Patent Literature 1, when a combination of multiple explanatory variables is taken to generate features to be used in data analysis, the number of features may become too large. In response to this, Patent Literature 1 describes the generation of features whose correlation coefficient with the objective variable is equal to or greater than a threshold value.

[0003] Patent Document 2 also describes a method of calculating an index value, which is the degree of co-occurrence in the data, for each combination of conditions related to multiple item values ​​contained in the data, using a model that has been trained on the data, and extracting specific combinations based on the conditions or index values. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-13511 A [Patent Document 2] JP 2020-140581 A Summary of the Invention [Problem to be solved by the invention]

[0005] One of the methods for evaluating an evaluation target is a method of evaluating the evaluation target using multiple indexes. The present inventors have considered that by combining two or more specific indexes among these multiple indexes, the evaluation target can be evaluated with high accuracy. The object of the present invention is to make it easier to select two or more indexes to be combined when evaluating an evaluation target. [Means for solving the problem]

[0006] According to the present invention, there is provided an acquisition means for acquiring categorical variable designation information for designating at least one of a plurality of indexes related to an evaluation target as a categorical variable; a selection means for selecting, for each of the categorical variables, a supporting variable that is a part of the plurality of indices; Equipped with The selection means is performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; An index selection device is provided that obtains a first influence degree indicating the magnitude of the influence of each of the multiple combination variables on the accuracy of the first model, and selects the support variable from the multiple indexes using the first influence degree.

[0007] According to the present invention, a computer An acquisition process of acquiring categorical variable designation information for designating at least one of a plurality of indicators related to the evaluation target as a categorical variable; a selection process for selecting, for each of the categorical variables, a supporting variable that is part of the plurality of indices; Do the following: In the selection process, the computer performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; An index selection method is provided, which obtains a first influence level indicating the magnitude of the influence of each of the multiple combination variables on the accuracy of the first model for each of the multiple combination variables, and selects the support variable from the multiple indexes using the first influence level.

[0008] According to the present invention, a computer includes: An acquisition function for acquiring categorical variable designation information for designating at least one of a plurality of indicators related to an evaluation target as a categorical variable; A selection function for selecting a supporting variable that is part of the plurality of indicators for each of the categorical variables; Let them have The selection function is performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; A program is provided that obtains a first influence degree indicating the magnitude of the influence of each of the multiple combination variables on the accuracy of the first model, and selects the support variable from the multiple indexes using the first influence degree. Effect of the Invention

[0009] According to the present invention, when evaluating an evaluation target, it becomes easier to select two or more indexes to be combined. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining a usage environment of an index selection device and an evaluation device. [Diagram 2] FIG. 2 illustrates an example of a functional configuration of an index selection device. [Diagram 3] FIG. 4 is a diagram illustrating an example of data stored in a training data storage unit. [Figure 4] 11 is a diagram showing an example of combination specifying information stored in a combination specifying information storage unit; FIG. [Diagram 5] 13 is a diagram showing an example of group specifying information stored in a combination specifying information storage unit; FIG. [Figure 6] FIG. 2 illustrates an example of a functional configuration of an evaluation device. [Figure 7] 4 is a diagram showing a first example of data stored in a target data storage unit; FIG. [Figure 8] FIG. 11 is a diagram showing a second example of data stored in the target data storage unit. [Figure 9] FIG. 11 is a diagram illustrating an example of data output by an output unit. [Figure 10]FIG. 2 is a diagram illustrating an example of a hardware configuration of the index selection device. [Figure 11] 13 is a flowchart illustrating an example of a process performed by an index selection device. [Figure 12] FIG. 12 is a diagram showing a modification of step S110 shown in FIG. [Figure 13] 4 is a flowchart showing a first example of processing performed by the evaluation device. [Figure 14] 14 is a flowchart showing details of step S262 in FIG. 13. [Figure 15] FIG. 14 is a diagram showing a modification of FIG. 13. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are given the same reference numerals and the description will be omitted as appropriate.

[0012] 1 is a diagram for explaining a usage environment of an index selection device 10 and an evaluation device 20 according to an embodiment. The evaluation device 20 generates information indicating an evaluation result of an evaluation target (hereinafter, referred to as evaluation information). The evaluation device 20 uses the evaluation model generated by the index selection device 10 when generating the evaluation information.

[0013] In detail, the state of the evaluation target is indicated by a plurality of indicators. For example, when the evaluation target is a company and the evaluation result indicates the soundness of the company (for example, bankruptcy probability), each of the plurality of indicators is a financial indicator. Also, when the evaluation target is an individual and the evaluation result indicates the creditworthiness of the individual, at least one of the plurality of indicators is the credit information, income, or deposit amount of the individual.

[0014] The evaluation model generated by the index selection device 10 generates evaluation information using an index that combines a categorical variable and a supporting variable. Hereinafter, a combination of a categorical variable and a supporting variable will be referred to as a combined variable. The categorical variable and the supporting variable are both one of the above-mentioned multiple indexes. In detail, the categorical variable is a main index, and the supporting variable is an index (however, an index different from the categorical variable) that is referred to when interpreting whether the value of the categorical variable is good or bad. For example, when the evaluation target is a company and the categorical variable is a safety index (e.g., debt dependency ratio), an example of the supporting variable is a profitability index (e.g., return on assets: ROA).

[0015] The number of categorical variables used in the evaluation model may be one, but is preferably multiple. However, the number of categorical variables is less than the number of the above-mentioned indicators. And, for each categorical variable, one or more supporting variables (preferably multiple supporting variables) are set.

[0016] 2 is a diagram showing an example of the functional configuration of the index selection device 10. The index selection device 10 has an acquisition unit 110 and a selection unit 120, and generates at least one (preferably a plurality of) the above-mentioned combination variables.

[0017] The acquiring unit 110 acquires information for designating categorical variables (hereinafter, referred to as categorical variable designation information). The acquiring unit 110 may acquire the categorical variable designation information from outside the index selection device 10, or may generate the categorical variable designation information by processing data. In the former case, the acquiring unit 110 may acquire the categorical variable designation information by user input, or may acquire the categorical variable designation information by communication. In the latter case, the acquiring unit 110 generates the categorical variable designation information by using, for example, data stored in the training data storage unit 130 described later.

[0018] The selection unit 120 selects a support variable from the above-mentioned multiple indices for each categorical variable. In other words, the selection unit 120 selects at least one (preferably multiple) combination variables, and generates information for identifying the selected combination variable (hereinafter, referred to as combination identification information). Set Specifically, the selection unit 120 executes machine learning with the above-mentioned combination variables as explanatory variables and the evaluation result of the evaluation target as a target variable to generate a first model. An example of the method used here is XGboost or dotDATA. Then, a first influence is generated for each of a plurality of combination variables. The first influence indicates the magnitude of the influence that the combination variable has on the accuracy of the first model. Then, the selection unit 120 selects support variables using this first influence. For example, the selection unit 120 selects a necessary number of support variables in descending order of the first influence. An example of the first influence is Feature Importance.

[0019] The selection unit 120 may generate the combination specification information by selecting a combination variable whose first influence degree satisfies a criterion among a plurality of combination variables, without selecting a support variable for each categorical variable. An example of the criterion used here is that "the first influence degree is greater than a criterion value", but it may also be that "the first influence degree is within a predetermined rank counting from the top".

[0020] In the example shown in FIG. 2, the index selection device 10 further includes a training data storage unit 130, a combination specification information storage unit 140, and a transmission unit 150.

[0021] The training data storage unit 130 stores training data for generating the first model. The selection unit 120 generates the first model using the training data stored in the training data storage unit 130, and generates a first influence degree for each of the multiple combination variables.

[0022] As described above, the training data has the combination variables as explanatory variables and the evaluation results of the evaluation targets as objective variables. The training data storage unit 130 does not need to store the training data itself, but only needs to store data necessary for generating the combination variables (e.g., the values ​​of each of the multiple indexes) and the evaluation results. In this case, the selection unit 120 generates the training data using the data stored in the training data storage unit 130.

[0023] The combination specification information storage unit 140 stores the above-mentioned combination specification information. This combination specification information is information indicating the evaluation model used by the evaluation device 20.

[0024] As will be described later in detail, a plurality of groups are defined according to the values ​​of the categorical variables. The evaluation device 20 changes the evaluation method of the evaluation object according to the group to which the evaluation object belongs. The combination specification information storage unit 140 also stores information for specifying these groups (hereinafter, referred to as group specification information).

[0025] The transmitting unit 150 transmits the combination specifying information and the group specifying information stored in the combination specifying information storage unit 140 to the evaluation device 20 .

[0026] FIG. 3 is a diagram showing an example of data stored in the training data storage unit 130. As described above, the training data storage unit 130 stores training data or information on which the training data is based. In the example shown in this figure, the training data storage unit 130 stores a plurality of data sets on which the training data is based. FIG. 3 shows one data set. As shown in FIG. 3, each of the plurality of data sets includes the values ​​of a plurality of indicators and the value of the evaluation result. One data set is actual data for an object of the same type as the evaluation target. For example, when the evaluation target is a company, one data set includes the values ​​of a plurality of financial indicators (corresponding to the values ​​of the plurality of indicators) for a certain company and data showing whether or not the company went bankrupt after a certain period of time as a binary value (corresponding to the value of the evaluation result). Also, when the evaluation target is an individual, one data set includes the values ​​of a plurality of indicators included in credit information for a certain individual and a value indicating creditworthiness (corresponding to the value of the evaluation result).

[0027] 4 is a diagram showing an example of combination specifying information stored in the combination specifying information storage unit 140. The combination specifying information includes information indicating indices that are categorical variables and information indicating indices that are supporting variables for the categorical variables. Here, multiple supporting variables may be specified for one categorical variable. Furthermore, when there are multiple categorical variables, the combination specifying information includes information indicating indices that are supporting variables for each categorical variable.

[0028] In addition, when there are multiple types of evaluation targets, the combination specific information storage unit 140 stores combination specific information for each type. For example, the combination specific information storage unit 140 stores combination specific information for evaluating a company and combination specific information for evaluating an individual.

[0029] 5 is a diagram showing an example of group identification information stored in the combination identification information storage unit 140. As described above, a plurality of groups are defined according to the values ​​of the categorical variables. The group identification information stores, for each group, information indicating a range of values ​​that defines the group. When a plurality of categorical variables are set, the combination identification information storage unit 140 stores group identification information for each of the plurality of categorical variables.

[0030] 6 is a diagram showing an example of a functional configuration of the evaluation device 20. The evaluation device 20 includes a combination acquisition unit 210 and an evaluation unit 230.

[0031] The combination acquisition unit 210 acquires combination specifying information and group specifying information from the index selection device 10, and stores them in the combination storage unit 220. The combination storage unit 220 may be a part of the evaluation device 20, or may be provided outside the evaluation device 20.

[0032] The evaluation section 230 performs processing using the combination specifying information and group specifying information stored in the combination storage section 220 to generate information indicating the evaluation results of each of the multiple supporting variables, that is, the above-mentioned evaluation information.

[0033] In detail, as described above, a plurality of groups are defined based on the values ​​of the categorical variables. Then, for each of the plurality of groups, a statistical value of the supporting variables is generated. This statistical value is generated for a population of evaluation targets whose categorical variable values ​​belong to the group. This statistical value is, for example, the average value of the supporting variables, but may be a value based on other statistical methods. Then, for each supporting variable, the evaluation unit 230 generates evaluation information, for example, an evaluation value, using the value of the supporting variable of the evaluation target and the statistical value of the supporting variable of the group to which the value of the categorical variable of the evaluation target belongs. An example of the evaluation value is a standardized value.

[0034] Below, an example will be given in which the evaluation target is a company, the evaluation result is the bankruptcy probability, and the categorical variables and supporting variables are financial indicators.

[0035] Each of the multiple groups is defined as a range of values ​​of financial indicators designated as categorical variables. For example, in the example shown in FIG. 5, multiple groups are defined by the value of the debt dependency. In addition, multiple financial indicators are designated as support variables for each of the categorical variables. The evaluation unit 230 acquires the value of the financial indicator designated as the categorical variable (e.g., the value of the debt dependency) from among the values ​​of each of the multiple financial indicators of the evaluation target, and identifies the group containing this value (i.e., the group to which the evaluation target belongs). For example, in the example shown in FIG. 5, when the debt dependency of the evaluation target company is 40%, the evaluation target belongs to group 2. Then, the evaluation unit 230 acquires the statistical values ​​of each of the multiple financial indicators designated as support variables from a population of multiple companies with a debt dependency of more than 35% and not more than 50%.

[0036] Furthermore, the evaluation unit 230 acquires the values ​​of the financial indicators designated as support variables from among the values ​​of the financial indicators of the evaluation target. The evaluation unit 230 then generates an evaluation value for each of the support variables (financial indicators) using the statistical values ​​of the support variables (financial indicators) in the specified group and the values ​​of the support variables (financial indicators) in the evaluation target.

[0037] When a plurality of combinations of categorical variables and a plurality of supporting variables are specified in the combination specification information, the evaluation section 230 generates evaluation information for each of the plurality of categorical variables and each of the plurality of supporting variables.

[0038] The process performed by the evaluation unit 230 will be described in detail later with reference to a flowchart.

[0039] The evaluation unit 230 acquires information on the evaluation target from the target data storage unit 240. The evaluation unit 230 also associates the generated evaluation information with the evaluation target and stores it in the target data storage unit 240. The target data storage unit 240 may be a part of the evaluation device 20 or may be provided outside the evaluation device 20.

[0040] The evaluation device 20 also includes an output unit 250. The output unit 250 generates output information using the evaluation information and outputs the output information. The output destination of the output information may be a display or a printing device. As described above, the categorical variables are main indicators, and the supporting variables are indicators that are referred to when interpreting the quality of the value of the categorical variables. The output information includes information indicating the type of the categorical variables (e.g., the name of a financial indicator) and the value of the categorical variables (e.g., the value of the financial indicator), information indicating the type of the supporting variables (e.g., the name of a financial indicator) and the value of the supporting variables (e.g., the value of the financial indicator), and evaluation information on the combination of the categorical variables and the supporting variables.

[0041] As described above, a plurality of supporting variables are associated with one categorical variable. For this reason, if information on all combinations of categorical variables and supporting variables (i.e., all combined variables) is included in the output information, the amount of information becomes too large. Therefore, the output unit 250 selects information on combined variables in which at least the supporting variables satisfy a predetermined criterion (hereinafter, referred to as the first criterion) and includes it in the output information. The first criterion is, for example, that when ranking based on evaluation information (for example, ranking based on the magnitude of the evaluation value) is performed on a plurality of combined variables, the combined variables are ranked within the first rank from the top or within the second rank from the bottom. This predetermined rank may be a predetermined rank of 3rd place or higher (including 1st place), or a predetermined rank of 5th place or higher. The first rank and the second rank may be the same or different. This predetermined rank may be set for each categorical variable.

[0042] FIG. 7 is a diagram showing a first example of data stored in the target data storage unit 240. As described above, the target data storage unit 240 stores information about the evaluation targets. Specifically, the target data storage unit 240 stores the values ​​of the indices of the evaluation targets for each evaluation target. The types of indices stored here are the same as the types of indices stored in the training data storage unit 130. For example, when the evaluation target is a company, the indices stored in the target data storage unit 240 are financial indices. Also, when the evaluation target is an individual, the indices stored in the target data storage unit 240 are indices included in credit information.

[0043] The target data storage unit 240 stores the values ​​of each index for each of the multiple evaluation targets. These values ​​are used when the evaluation unit 230 calculates the above-mentioned statistical values. However, the above-mentioned statistical values ​​may be calculated using the values ​​of each index for targets other than the evaluation targets that are the processing target of the evaluation unit 230. For example, the above-mentioned statistical values ​​may be calculated using financial indicators of companies other than the company that is the evaluation target, or may be calculated using credit information of individuals other than the individual that is the evaluation target. In these cases, the above-mentioned statistical values ​​may be further calculated using the statistical values ​​of the evaluation targets stored in the target data storage unit 240.

[0044] The statistical values ​​themselves may also be stored in the target data storage unit 240. In this case, the evaluation unit 230 reads out the values ​​of the indices of the evaluation target from the target data storage unit 240 and also reads out the statistical values.

[0045] 8 is a diagram showing a second example of data stored in the target data storage unit 240. As described above, the target data storage unit 240 stores the evaluation information generated by the evaluation unit 230. Specifically, the target data storage unit 240 stores evaluation information for each combination of categorical variables and supporting variables (i.e., for each combined variable) for each evaluation target. Here, it is preferable that the target data storage unit 240 stores evaluation information for all combined variables.

[0046] FIG. 9 is a diagram showing an example of data output by the output unit 250. In the example shown in this figure, the output unit 250 outputs information on a specific combination variable for a certain evaluation target. The output information includes evaluation information on the combination variable (described as index evaluation score in FIG. 9), as well as the names of categorical variables constituting the combination variable (described as "category name" in FIG. 9) and the values ​​of the categorical variables (described as "category value" in FIG. 9), and the names of supporting variables constituting the combination variable and the values ​​of the supporting variables. Furthermore, in the example of FIG. 9, the output unit 250 also outputs information indicating which group the evaluation target belongs to with respect to the categorical variable (described as "threshold condition" in the example shown in FIG. 9).

[0047] The output unit 250 also outputs a sentence indicating the interpretation result of the combined variable. This sentence is generated using a model that takes as input, for example, evaluation information of the combined variable, the names of categorical variables constituting the combined variable and the values ​​of the categorical variables, and the names of supporting variables constituting the combined variable and the values ​​of the supporting variables, and outputs the sentence. This model may be generated using machine learning. Note that this sentence may be input by a person who has seen the result shown in FIG. 9.

[0048] In the example shown in FIG. 9, the output unit 250 outputs auxiliary evaluation information for each combination variable. This auxiliary evaluation information is generated by the evaluation unit 230 and stored in the target data storage unit 240. The evaluation unit 230 sets the support variable to the second categorical variable, and sets a plurality of indices different from the categorical variables and support variables constituting the combination variable to the second support variable. Then, the evaluation unit 230 performs a process similar to the process for the combination variable described above for the second categorical variable and the second support variable, thereby generating evaluation information for the second support variable as the auxiliary evaluation information described above. The second support variable is an index to be referred to when interpreting whether the value of the support variable is good or bad. Then, the output unit 250 selects the second support variable with the highest auxiliary evaluation information (for example, the highest evaluation value) for each combination variable, and outputs the name of the second support variable (described as "auxiliary variable name" in FIG. 9) together with the value of the second support variable (described as "auxiliary variable value" in FIG. 9). Here, the output unit 250 may further output evaluation information for the second support variable.

[0049] For example, assume that the evaluation target is a company, the categorical variable is the debt dependency, the supporting variable is the tangible fixed asset turnover, and the second supporting variable is the liquidity ratio. If the debt dependency is, for example, 25% and the tangible fixed asset turnover is 5 times, it is generally determined that the profitability is sufficiently high even if the debt is small. Here, if the liquidity ratio is high, for example, 300%, it can be interpreted that there is a low possibility of a short-term capital shortage. On the other hand, if the liquidity ratio is low, for example, 100%, it can be interpreted that the necessary working capital cannot be secured, and therefore the profitability may appear high due to bad debts and dead inventory. Even in this case, if there is no problem with, for example, accounts receivable and inventory, it may be determined that the cash shortage is temporary and that the low debt is a good condition.

[0050] It should be noted that the evaluation unit 230 does not have to generate this auxiliary evaluation information.

[0051] In addition, since there are many types of combination variables, as described above, the output unit 250 selects combination variables that satisfy the first criterion as combination variables for which information should be output. As an example, when ranking a plurality of combination variables based on evaluation information, the output unit 250 selects combination variables that are within the first rank from the top and selects combination variables that are within the second rank from the bottom. Then, the output unit 250 outputs the above-mentioned information of the selected combination variables.

[0052] In addition, when there is a specified input (for example, an input indicating a desire to check the evaluation information of other combination variables), the output section 250 outputs information for support variables whose evaluation information does not satisfy the first criterion similar to that for support variables whose evaluation information satisfies the first criterion.

[0053] In addition, the output unit 250 selects a supporting variable whose evaluation information satisfies the second criterion for each of the multiple categorical variables, and outputs the selected supporting variable and the categorical variable as Out In this case, the output section 250 selects the combination variable to be output from the selected candidates.

[0054] As mentioned above, supporting variables are indicators that are referenced when interpreting the value of a categorical variable. Therefore, if supporting variables with conflicting interpretations are selected, there is a possibility that confusion may occur in the interpretation of the categorical variable. For this reason, the second criterion is set to bring consistency to the selected supporting variables. For example, when the evaluation information is a standardized value, the second criterion is that the supporting variable with the largest absolute value of the standardized value and the evaluation information are the same in positive and negative sign. For example, when the standardized value of the supporting variable with the largest absolute value of the standardized value is a negative value, the second criterion is that "the evaluation information is negative."

[0055] Another example of the second criterion is that the same index is not used as a supporting variable a certain number of times (e.g., three times or more) when looking at multiple combined variables as a whole. This is because if a certain index is an abnormal value, the index is likely to be repeatedly selected as a supporting variable, and the evaluation of the evaluation target will be significantly influenced by the abnormal value.

[0056] Furthermore, the evaluation unit 230 uses the evaluation results of each of the multiple combined variables to generate information indicating the overall evaluation result of the evaluation target (hereinafter, referred to as integrated evaluation information). For example, when the evaluation results of each of the multiple combined variables have a score, the integrated evaluation information is the result of statistically processing these scores. For example, when the evaluation results of each combined variable include a standardized value, the integrated evaluation information is generated by adding up all of these standardized values.

[0057] 10 is a diagram showing an example of a hardware configuration of the index selection device 10. The index selection device 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0058] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0059] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0060] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0061] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the index selection device 10 (e.g., the acquisition unit 110, the selection unit 120, and the transmission unit 150). The processor 1020 loads each of these program modules onto the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 also functions as the training data storage unit 130 and the combination identification information storage unit 140.

[0062] The input / output interface 1050 is an interface for connecting the index selection device 10 to various input / output devices.

[0063] The network interface 1060 is an interface for connecting the index selection device 10 to a network. This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1060 may connect to the network wirelessly or by wire. The index selection device 10 may communicate with the evaluation device 20 via the network interface 1060.

[0064] The hardware configuration of the evaluation device 20 is similar to that of the index selection device 10 shown in Fig. 10. The storage device 1040 stores program modules that realize each function of the evaluation device 20 (e.g., the combination acquisition unit 210, the evaluation unit 230, and the output unit 250). The storage device 1040 also functions as the combination storage unit 220 and the target data storage unit 240.

[0065] 11 is a flowchart showing an example of processing performed by the index selection device 10. First, the acquisition unit 110 of the index selection device 10 acquires categorical variable designation information from outside the index selection device 10 (step S110). For example, when there are 200 types of indexes, the number of categorical variables designated by the categorical variable designation information is preferably 50 or less.

[0066] The selection unit 120 then selects one of the categorical variables specified by the categorical variable designation information (step S120). The selection unit 120 then generates a plurality of combined variables using the selected categorical variable (step S130). The number of combined variables generated here is the "number of indexes used - 1". For example, if there are 200 types of indexes, the number of combined variables generated here is 199.

[0067] The selection unit 120 then generates the first model described above using the multiple combination variables, and generates the first influence for each of the multiple combination variables (step S140). The selection unit 120 then uses these first influences to select support variables for the categorical variables selected in step S120 (step S150). This selects the combination variables. The selection unit 120 then stores information indicating the selected combination variables as combination identification information in the combination identification information storage unit 140 (step S160).

[0068] The index selection device 10 performs the processes shown in steps S130 to S160 for all categorical variables indicated by the categorical variable designation information (steps S170 and S120).

[0069] The selection unit 120 may perform the processes shown in steps S150 and S160 after step S170. In this case, the selection unit 120 calculates the first influence degree for all combination variables, and then generates combination specification information by selecting a predetermined number of combination variables in descending order of the first influence degree. identificationThe information is stored in the combination specific information storage unit 140. In this case, the number of supporting variables may differ for each categorical variable. For example, there may be categorical variables for which multiple supporting variables are selected, and there may be categorical variables for which no supporting variables are selected at all. In this case, the combination variables may be further narrowed down in consideration of multicollinearity. This narrowing down may be performed, for example, by a person or by the selection unit 120.

[0070] Furthermore, the index selection device 10 generates group identifying information separately from the process shown in the figure, and stores the generated group identifying information in the combination identifying information storage unit 140. Thereafter, the transmission unit 150 of the index selection device 10 transmits the combination identifying information and the group identifying information to the evaluation device 20 at a predetermined timing.

[0071] Fig. 12 is a diagram showing a modified example of step S110 shown in Fig. 11. In this modified example, the acquiring unit 110 processes the information stored in the training data storage unit 130 to generate categorical variable designation information.

[0072] Specifically, the acquiring unit 110 generates a second model using data stored in the training data storage unit 130. This second model is generated by executing machine learning with a plurality of indices as explanatory variables and an evaluation result as a target variable. An example of a method used here is XGboost or dotDATA. The acquiring unit 110 also generates a second influence for each of the plurality of indices. The second influence is, for example, Feature Importance, and indicates the magnitude of the influence that the indices have on the accuracy of the second model (step S112).

[0073] Then, the acquiring section 110 selects a categorical variable from the multiple indices using the second influence. For example, the acquiring section 110 selects a predetermined number of indices as categorical variables in descending order of the second influence (step S114).

[0074] Fig. 13 is a flowchart showing a first example of processing performed by the evaluation device 20. The processing shown in this figure is processing for evaluating the evaluation target. Prior to this processing, the combination acquisition unit 210 of the evaluation device 20 acquires combination specifying information and group specifying information from the index selection device 10 and stores them in the combination storage unit 220.

[0075] First, the evaluation unit 230 reads out combination specification information from the combination storage unit 220 (step S210). The combination specification information indicates categorical variables and supporting variables used in the categorical variables. The evaluation unit 230 also reads out data of each index to be evaluated from the target data storage unit 240 (step S220). Then, the evaluation unit 230 performs the processes shown in steps S240 to S262 for each of the categorical variables indicated by the combination specification information (steps S230 and S270).

[0076] First, the evaluation unit 230 selects one categorical variable from the multiple categorical variables indicated by the combination identification information (step S230). This selection also determines multiple indicators to be used as supporting variables. Then, the evaluation unit 230 reads out group identification information for the selected categorical variable from the combination storage unit 220 (step S240). Then, the evaluation unit 230 uses the group identification information to identify a group to which the value of the categorical variable to be evaluated belongs. Then, the evaluation unit 230 obtains statistical values ​​of each supporting variable in the group (step S250). Then, the evaluation unit 230 uses these statistical values ​​and the values ​​of each supporting variable to be evaluated to generate evaluation information for each supporting variable, and stores the evaluation information in the target data storage unit 240 (step S260).

[0077] Furthermore, the evaluation unit 230 generates auxiliary evaluation information for each supporting variable, and stores it in the target data storage unit 240 (step S262). Details of step S262 will be described later with reference to FIG.

[0078] Then, the evaluation unit 230 generates integrated evaluation information and stores it in the target data storage unit 240 (step S280). After that, the output unit 250 generates output information (for example, screen data and / or print data) and outputs this output information (step S290).

[0079] Fig. 14 is a flowchart showing details of step S262 in Fig. 13. The evaluation unit 230 performs the processes shown in steps S302 to S306 for each of the multiple support variables (that is, multiple second categorical variables) (steps S300 and S308).

[0080] First, the evaluation unit 230 selects one support variable (second categorical variable) (step S300). This selection also determines a number of indices to be used as the second support variables. The evaluation unit 230 then reads out group identification information for the selected second categorical variable from the combination storage unit 220 (step S302). The evaluation unit 230 then uses the group identification information to identify the group to which the value of the second categorical variable of the evaluation target belongs. The evaluation unit 230 then obtains statistical values ​​for each second support variable in the group (step S304). The evaluation unit 230 then uses these statistical values ​​and the values ​​of each second support variable of the evaluation target to generate evaluation information (i.e., auxiliary evaluation information) for each second support variable, and stores the information in the target data storage unit 240 (step S306).

[0081] Fig. 15 is a diagram showing a modified example of Fig. 13. This modified example is similar to the example shown in Fig. 14, except that for a group that satisfies the third criterion, evaluation information of supporting variables is not generated.

[0082] Specifically, the evaluation unit 230 evaluates the group identification After obtaining the information (step S240), and before calculating the statistical values ​​of the support variables (step S250), the group that does not satisfy the third criterion is selected. P, and selects it as a group for which evaluation information should be generated (step S242). Then, the evaluation unit 230 performs the processes shown in steps S260 and S262 only if the value of the categorical variable to be evaluated belongs to the group selected in step S242 (step S244: Yes).

[0083] The third criterion indicates, for example, that the group has a standard value of the categorical variable. If the value of the categorical variable of the evaluation target belongs to a group that satisfies the third criterion, the evaluation target is standard with respect to the categorical variable, and therefore the evaluation result of the evaluation target is not significantly affected. Therefore, for a group that satisfies the third criterion, the evaluation result of the evaluation target is almost unchanged even if evaluation information of the support variables is not generated. An example of the third criterion is that the group includes at least one of the mode, median, and mean of the categorical variable.

[0084] If the value of the categorical variable to be evaluated does not belong to the group selected in step S242 (step S244: No), the evaluation unit 230 may assign a fixed value as the evaluation value of the support variable in step S260. This fixed value is a standard value (for example, the average, median, or mode) as an evaluation value.

[0085] As described above, according to this embodiment, by using the index selection device 10, it becomes easier to select two indexes to be combined when evaluating an evaluation target. Furthermore, the evaluation device 20 generates evaluation information for each of the two indexes (i.e., combined variables) selected by the index selection device 10 for the evaluation target. Therefore, by checking this evaluation information, the user of the evaluation device 20 can more easily evaluate the evaluation target with high accuracy.

[0086] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0087] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the figures can be changed to the extent that the content is not affected. In addition, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0088] A part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. An acquisition means for acquiring categorical variable designation information for designating at least one of a plurality of indicators related to an evaluation target as a categorical variable; a selection means for selecting, for each of the categorical variables, a supporting variable that is a part of the plurality of indices; Equipped with The selection means is performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; An index selection device that obtains a first influence degree indicating the magnitude of the influence that the combination variable has on the accuracy of the first model for each of the multiple combination variables, and selects the support variable from the multiple indexes using the first influence degree. 2. In the index selection device according to the above item 1, The acquisition means includes: performing machine learning using the plurality of indexes as explanatory variables and the evaluation result as a response variable to generate a second model; An index selection device that obtains a second influence degree indicating the magnitude of the influence of each of the multiple indexes on the accuracy of the second model, and selects the categorical variable from the multiple indexes using the second influence degree. 3. In the index selection device according to 1 or 2 above, the categorical variable designation information designates a plurality of the categorical variables, The selection means is generating the plurality of combined variables, generating a first model, and acquiring the first influence degree for each of the plurality of categorical variables; The indicator selection device selects the supporting variables by selecting the combination variables whose first influence degree satisfies a criterion. 4. In the index selection device according to any one of 1 to 3 above, The evaluation target is a company, An index selection device, wherein each of the plurality of indexes is a financial index. 5. In the index selection device according to any one of 1 to 3 above, The subject of evaluation is an individual, An indicator selection device, wherein at least one of the multiple indicators is the individual's credit information, income, or deposit amount. 6. The computer An acquisition process of acquiring categorical variable designation information for designating at least one of a plurality of indicators related to the evaluation target as a categorical variable; a selection process for selecting, for each of the categorical variables, a supporting variable that is part of the plurality of indices; Do the following: In the selection process, the computer performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; An index selection method comprising: obtaining a first influence degree indicating the magnitude of an influence that the combination variable has on the accuracy of the first model for each of the plurality of combination variables; and selecting the support variable from the plurality of indexes using the first influence degree. 7. In the index selection method described in 6 above, In the acquisition process, the computer performing machine learning using the plurality of indexes as explanatory variables and the evaluation result as a response variable to generate a second model; An indicator selection method comprising: obtaining a second influence degree indicating the magnitude of the influence of each of the plurality of indicators on the accuracy of the second model; and selecting the categorical variable from the plurality of indicators using the second influence degree. 8. In the index selection method according to 6 or 7 above, the categorical variable designation information designates a plurality of the categorical variables, In the selection process, the computer generating the plurality of combined variables, generating a first model, and acquiring the first influence degree for each of the plurality of categorical variables; The indicator selection method selects the supporting variables by selecting the combination variables whose first influence degree satisfies a criterion. 9. In the index selection method according to any one of 6 to 8 above, The evaluation target is a company, The method of claim 1, wherein each of the plurality of metrics is a financial metric. 10. In the index selection method according to any one of 6 to 8 above, The subject of evaluation is an individual, An indicator selection method, wherein at least one of the multiple indicators is the individual's credit information, income, or deposit amount. 11. On the computer, An acquisition function for acquiring categorical variable designation information for designating at least one of a plurality of indicators related to an evaluation target as a categorical variable; A selection function for selecting a supporting variable that is part of the plurality of indicators for each of the categorical variables; Let them have The selection function is performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; A program that acquires a first influence degree indicating the magnitude of an influence that the combination variable has on the accuracy of the first model for each of the plurality of combination variables, and selects the support variable from the plurality of indicators using the first influence degree. 12. In the program according to 11 above, The acquisition function includes: performing machine learning using the plurality of indexes as explanatory variables and the evaluation result as a response variable to generate a second model; A program that obtains a second influence degree indicating the magnitude of the influence of each of the plurality of indexes on the accuracy of the second model, and selects the categorical variable from the plurality of indexes using the second influence degree. 13. In the program according to 11 or 12 above, the categorical variable designation information designates a plurality of the categorical variables, The selection function is generating the plurality of combined variables, generating a first model, and acquiring the first influence degree for each of the plurality of categorical variables; The program selects the supporting variables by selecting the combination variables whose first influence satisfies a criterion. 14. In the program according to any one of the above 11 to 13, The evaluation target is a company, Each of the plurality of indicators is a financial indicator. 15. In the program according to any one of the above 11 to 13, The subject of evaluation is an individual, At least one of the plurality of indicators is the individual's credit information, income, or deposit amount.

[0089] This application claims priority based on Japanese Patent Application No. 2021-010113, filed on January 26, 2021, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0090] 10. Index selection device 20 Evaluation Equipment 110 Acquisition Department 120 Selection section 130 Training data storage unit 140 Combination specific information storage unit 150 Transmitter 210 Combination Acquisition Unit 220 Combination memory unit 230 Evaluation Department 240 Target data storage unit 250 Output section

Claims

1. An acquisition means for acquiring categorical variable designation information for designating at least one of a plurality of indexes related to an evaluation target as a categorical variable; a selection means for selecting, for each of the categorical variables, a supporting variable that is a part of the plurality of indices; Equipped with The selection means is performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; An index selection device that obtains a first influence degree indicating the magnitude of the influence that the combination variable has on the accuracy of the first model for each of the multiple combination variables, and selects the support variable from the multiple indexes using the first influence degree.

2. 2. The index selection device according to claim 1, The acquisition means includes: performing machine learning using the plurality of indexes as explanatory variables and the evaluation result as a response variable to generate a second model; An index selection device that obtains a second influence degree indicating the magnitude of the influence of each of the multiple indexes on the accuracy of the second model, and selects the categorical variable from the multiple indexes using the second influence degree.

3. The index selection device according to claim 1 or 2, the categorical variable designation information designates a plurality of the categorical variables, The selection means is generating the plurality of combined variables, generating a first model, and acquiring the first influence degree for each of the plurality of categorical variables; An index selection device that selects the supporting variables by selecting the combination variables whose first influence degree satisfies a criterion.

4. In the index selection device according to any one of claims 1 to 3, The evaluation target is a company, An index selection device, wherein each of the plurality of indexes is a financial index.

5. In the index selection device according to any one of claims 1 to 3, The subject of evaluation is an individual, An indicator selection device, wherein at least one of the multiple indicators is the individual's credit information, income, or deposit amount.

6. The computer An acquisition process of acquiring categorical variable designation information for designating at least one of a plurality of indicators related to the evaluation target as a categorical variable; a selection process for selecting, for each of the categorical variables, a supporting variable that is part of the plurality of indices; Do the following: In the selection process, the computer performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; An index selection method comprising: obtaining a first influence degree indicating the magnitude of the influence of each of the plurality of combination variables on the accuracy of the first model; and selecting the support variable from the plurality of indexes using the first influence degree.

7. The index selection method according to claim 6, In the acquisition process, the computer performing machine learning using the plurality of indexes as explanatory variables and the evaluation result as a response variable to generate a second model; An index selection method comprising: obtaining a second influence degree indicating the magnitude of the influence of each of the plurality of indexes on the accuracy of the second model; and selecting the categorical variable from the plurality of indexes using the second influence degree.

8. The index selection method according to claim 6 or 7, the categorical variable designation information designates a plurality of the categorical variables, In the selection process, the computer generating the plurality of combined variables, generating a first model, and acquiring the first influence degree for each of the plurality of categorical variables; The indicator selection method selects the supporting variables by selecting the combination variables whose first influence degree satisfies a criterion.

9. On the computer, An acquisition function for acquiring categorical variable designation information for designating at least one of a plurality of indicators related to an evaluation target as a categorical variable; A selection function for selecting a supporting variable that is part of the plurality of indicators for each of the categorical variables; Let them have The selection function is performing machine learning using the categorical variable and a combination variable, which is a combination of the indexes different from the categorical variable, as explanatory variables and the evaluation result of the evaluation target as an objective variable to generate a first model; A program that obtains a first influence degree indicating the magnitude of an influence that the combination variable has on the accuracy of the first model for each of the multiple combination variables, and selects the support variable from the multiple indicators using the first influence degree.

10. The program according to claim 9, The acquisition function includes: performing machine learning using the plurality of indexes as explanatory variables and the evaluation result as a response variable to generate a second model; A program that obtains a second influence degree indicating the magnitude of the influence of each of the plurality of indexes on the accuracy of the second model, and selects the categorical variable from the plurality of indexes using the second influence degree.

11. In the index selection device according to any one of claims 1 to 5, The selection means selects combination variables from the plurality of combination variables with a higher magnitude of the first influence, and selects the different indicators included in the selected combination variables as the supporting variables.

12. In the index selection device according to claim 11, The selection means is configured to preferentially select, from among the plurality of combination variables, a combination variable whose first influence degree is greater than a reference value.

13. In the index selection device according to claim 11, The selection means is an index selection device that preferentially selects a combination variable that is within a predetermined rank, counting from the top, in an order of the multiple combination variables that is determined based on the magnitude of the first influence degree.

14. The index selection method according to any one of claims 6 to 8, The index selection method, in which the computer, in the selection process, prioritizes the selection of combination variables having a greater magnitude of the first influence among the multiple combination variables, and selects the different indexes included in the selected combination variables as the supporting variables.

15. In the index selection method according to claim 14, The index selection method, wherein the computer, in the selection process, preferentially selects, from the plurality of combined variables, a combined variable whose first influence degree is greater than a reference value.

16. In the index selection method according to claim 14, The index selection method, wherein the computer, in the selection process, preferentially selects combination variables that are within a predetermined rank, counting from the top, in an order of the multiple combination variables that is determined based on the magnitude of the first influence.

17. In the program according to claim 9 or 10, The selection function is a program that prioritizes selection of combination variables among the multiple combination variables in accordance with the magnitude of the first influence, and selects the different indexes included in the selected combination variables as the support variables.

18. In the program according to claim 17, The selection function is a program for preferentially selecting, from among the plurality of combination variables, a combination variable whose first influence degree is greater than a reference value.

19. In the program according to claim 17, The selection function is a program for preferentially selecting a combination variable that is within a predetermined rank, counting from the top, in an order of the multiple combination variables determined based on the magnitude of the first influence degree.

Images